Ms Manual
User Manual:
Open the PDF directly: View PDF .
Page Count: 348
Download | |
Open PDF In Browser | View PDF |
Rational Software Corporation® RATIONAL® CLEARCASE MULTISITE® ADMINISTRATOR’S GUIDE VERSION: 2002.05.00 AND LATER PART NUMBER: 800-025073-000 UNIX/WINDOWS EDITION Administrator’s Guide Document Number 800-025073-000 October 2001 Rational Software Corporation 20 Maguire Road Lexington, Massachusetts 02421 IMPORTANT NOTICE Copyright Copyright © 1992, 2001 Rational Software Corporation. All rights reserved. Copyright 1989, 1991 The Regents of the University of California Copyright 1984–1991 by Raima Corporation Permitted Usage THIS DOCUMENT IS PROTECTED BY COPYRIGHT AND CONTAINS INFORMATION PROPRIETARY TO RATIONAL. ANY COPYING, ADAPTATION, DISTRIBUTION, OR PUBLIC DISPLAY OF THIS DOCUMENT WITHOUT THE EXPRESS WRITTEN CONSENT OF RATIONAL IS STRICTLY PROHIBITED. THE RECEIPT OR POSSESSION OF THIS DOCUMENT DOES NOT CONVEY ANY RIGHTS TO REPRODUCE OR DISTRIBUTE ITS CONTENTS, OR TO MANUFACTURE, USE, OR SELL ANYTHING THAT IT MAY DESCRIBE, IN WHOLE OR IN PART, WITHOUT THE SPECIFIC WRITTEN CONSENT OF RATIONAL. Trademarks Rational, Rational Software Corporation, the Rational logo, Rational the e-development company, Rational Suite ContentStudio, ClearCase, ClearCase MultiSite ClearQuest, Object Testing, Object-Oriented Recording, Objectory, PerformanceStudio, PureCoverage, PureDDTS, PureLink, Purify, Purify'd, Quantify, Rational Apex, Rational CRC, Rational PerformanceArchitect, Rational Rose, Rational Suite, Rational Summit, Rational Unified Process, Rational Visual Test, Requisite, RequisitePro, RUP, SiteCheck, SoDA, TestFactory, TestMate, TestStudio, The Rational Watch, among others are trademarks or registered trademarks of Rational Software Corporation in the United States and in other countries. All other names are used for identification purposes only, and are trademarks or registered trademarks of their respective companies. Sun, Solaris, and Java are trademarks or registered trademarks of Sun Microsystems, Inc. Microsoft, the Microsoft logo, the Microsoft Internet Explorer logo, Windows, the Windows logo, Windows NT, the Windows Start logo are trademarks or registered trademarks of Microsoft Corporation in the United States and other countries. Patent U.S. Patent Nos. 5,574,898 and 5,649,200 and 5,675,802. Additional patents pending. Government Rights Legend Use, duplication, or disclosure by the U.S. Government is subject to restrictions set forth in the applicable Rational License Agreement and in DFARS 227.7202-1(a) and 227.7202-3(a) (1995), DFARS 252.227-7013(c)(1)(ii) (Oct 1988), FAR 12.212(a) 1995, FAR 52.227-19, or FAR 52.227-14, as applicable. Warranty Disclaimer This document and its associated software may be used as stated in the underlying license agreement. Rational Software Corporation expressly disclaims all other warranties, express or implied, with respect to the media and software product and its documentation, including without limitation, the warranties of merchantability or fitness for a particular purpose or arising from a course of dealing, usage, or trade practice. Technical Acknowledgments This software and documentation is based in part on BSD Networking Software Release 2, licensed from the Regents of the University of California. We acknowledge the role of the Computer Systems Research Group and the Electrical Engineering and Computer Sciences Department of the University of California at Berkeley and the Other Contributors in its development. This product includes software developed by Greg Steinfor use in the mod_dav module for Apache (http://www.webdav.org/mod_dav/). Contents Preface .................................................................................................................................... xvii About This Manual ....................................................................................... xvii ClearCase Documentation Roadmap ........................................................ xviii Typographical Conventions ..........................................................................xix Online Documentation ....................................................................................xx Technical Support ............................................................................................xx MultiSite Overview 1. Introduction to MultiSite .................................................................................................... 1 1.1 VOBs and VOB Replicas ................................................................................... 1 Replica Names, Replica Objects, and Host Assignments.............................2 Differences Among Sites ................................................................................... 2 Element Ownership and Ownership Preservation ....................................... 4 1.2 Requirements for Ownership-Preserving Replicas................................5 Synchronizing Replicas in a VOB Family ....................................................... 6 MultiSite, Time, and Time Zones..................................................................... 6 Enabling Independent VOB Development: Mastership...............................7 Replica Mastership............................................................................................. 8 Branch Mastership ............................................................................................. 8 Creation of the main Branch of an Element .......................................... 10 Synchronizing Development on Different Branches ...........................10 1.3 1.4 1.5 Default and Explicit Branch Mastership................................................ 13 Type Object Mastership................................................................................... 14 Mastership Restrictions ................................................................................... 17 Supporting Serial Development in Replicas ................................................ 20 Conflict Resolution........................................................................................... 21 Resolving Conflicts Among Type Objects .................................................... 21 VOB Objects and Replica Objects .................................................................. 23 Contents iii /vobs/multisite_doc/manual/ms_manualTOC.fm — September 11, 2001 4:10 pm 1.6 VOB Operations and the Oplog .....................................................................24 Tracking Operations for Each Replica...........................................................25 Epoch Numbers ................................................................................................26 Optimization and the Epoch Number Matrix ..............................................27 Indirect Synchronization ..........................................................................29 2. Planning a MultiSite Implementation ............................................................................33 2.1 MultiSite Installation........................................................................................34 2.2 MultiSite Licensing...........................................................................................35 2.3 ClearCase Use Model .......................................................................................35 Branching and Mastership ..............................................................................36 Use of Metadata ................................................................................................37 Text Mode for Replicas ....................................................................................37 Use of Administrative VOBs or UCM ...........................................................38 2.4 MultiSite Use Model.........................................................................................38 Type of Administration ...................................................................................38 Mastership Strategy..........................................................................................40 Replica Permission Strategy............................................................................40 Synchronization Method .................................................................................41 Synchronization Pattern ..................................................................................42 Directions of Exchange .............................................................................42 One-to-One and Ring Synchronization..................................................43 One-to-Many Synchronization ................................................................44 Many-to-Many Synchronization .............................................................46 Synchronization Schedule ...............................................................................46 Use of MultiSite for Backups ..........................................................................47 Scrubbing Parameters for VOB Replicas.......................................................47 Oplog Scrubbing........................................................................................48 2.5 iv export_sync Scrubbing .............................................................................49 Responsibilities of MultiSite Administrators ...............................................49 Administrator’s Guide: Rational ClearCase MultiSite /vobs/multisite_doc/manual/ms_manualTOC.fm — September 11, 2001 4:10 pm 3. MultiSite Command Set.................................................................................................... 53 3.1 Location of MultiSite Programs ..................................................................... 53 3.2 multitool Use..................................................................................................... 54 multitool Subcommands ................................................................................. 55 Commands Copied from ClearCase ...................................................... 55 Replica Creation, Synchronization, and Management ........................55 Object Mastership ..................................................................................... 56 3.3 3.4 3.5 3.6 Failure Recovery........................................................................................ 57 View Contexts and VOB Mounts................................................................... 58 Specifying VOBs and Replicas in Commands ............................................. 58 Additional MultiSite Commands................................................................... 59 ClearCase Commands Related to MultiSite ................................................. 60 Using MultiSite 4. Creating Replicas............................................................................................................... 65 4.1 Overview of Replica Creation ........................................................................ 65 4.2 Timing of Replica Creation ............................................................................. 66 4.3 Notes on Different Transport Methods......................................................... 66 Store-and-Forward Method............................................................................ 66 Communication Between Replica Hosts ............................................... 66 Limiting the Size of a Packet ................................................................... 67 4.4 4.5 Transport Options..................................................................................... 67 Notes on Using Tape or a File-Based Transfer Method..............................67 Replica-Creation Scenario ............................................................................... 68 Planning the Rules of the Road ...................................................................... 68 Prerequisites...................................................................................................... 70 Export Phase ..................................................................................................... 71 Transport Phase................................................................................................ 73 Import Phase ..................................................................................................... 73 Replicating a VOB Between UNIX and Windows....................................... 77 Contents v /vobs/multisite_doc/manual/ms_manualTOC.fm — September 11, 2001 4:10 pm 5. ClearCase Feature Levels ................................................................................................79 5.1 Overview of Feature Levels ............................................................................79 5.2 Raising the Replica Feature Level ..................................................................80 5.3 Raising the VOB Family Feature Level .........................................................82 VOB Families with Bidirectional Synchronization ......................................82 VOB Families with Unidirectional Synchronization ...................................82 5.4 Displaying Feature Levels...............................................................................83 5.5 Feature Levels Error Message.........................................................................84 6. Synchronizing Replicas....................................................................................................85 6.1 Synchronization Patterns.................................................................................86 Designing an Update Strategy........................................................................87 6.2 Assumption of Successful Synchronization .................................................90 6.3 Transferring Packets with Store-and-Forward.............................................90 Packet Storage During the Export and Import Phases................................91 Packet Transport ...............................................................................................92 Configuring the Store-and-Forward Facility................................................92 Submitting Packets to Store-and-Forward....................................................92 Sending Files That Are Not Packets...............................................................93 Setting Up an Indirect Shipping Route .........................................................93 Retries, Expirations, and Returned Data.......................................................94 6.4 Error Notification in a Mixed Environment ..........................................95 Differentiating Packets with Storage Classes ...............................................95 Using MultiSite through a Firewall ...............................................................96 Using Electronic Mail.......................................................................................96 Using FTP ..........................................................................................................97 Using Custom Software...................................................................................98 Installing Store-and-Forward on a UNIX Firewall Host ............................98 Firewall Issues..........................................................................................100 Installing shipping_server on a Firewall .............................................100 Controlling Ports Used by albd_server and shipping_server ..........101 Guidelines for Setting Port Values........................................................101 Specifying Port Values............................................................................101 vi Administrator’s Guide: Rational ClearCase MultiSite /vobs/multisite_doc/manual/ms_manualTOC.fm — September 11, 2001 4:10 pm 6.5 6.6 Manual Synchronization ............................................................................... 102 Export Phase ................................................................................................... 102 Transport Phase.............................................................................................. 103 Import Phase ................................................................................................... 103 Automated Synchronization......................................................................... 104 Using the ClearCase Scheduler .................................................................... 104 Export Phase ................................................................................................... 105 Transport Phase.............................................................................................. 106 Import Phase ................................................................................................... 107 Defining Receipt Handlers .................................................................... 108 6.7 6.8 Scheduling Import Jobs.......................................................................... 108 Listing Synchronization History .................................................................. 109 Synchronizing More Efficiently ................................................................... 109 Example of Increased Efficiency .................................................................. 109 Example of Decreased Efficiency................................................................. 110 7. Managing Replicas .......................................................................................................... 111 7.1 Displaying Properties of a Replica .............................................................. 111 7.2 Listing the Synchronization History of a Replica...................................... 113 7.3 Changing the Host Name for a Replica ...................................................... 113 7.4 Changing Ownership Preservation ............................................................. 114 7.5 Setting the Connectivity Property ............................................................... 116 7.6 Renaming a Replica ....................................................................................... 116 7.7 Moving a Replica............................................................................................ 117 7.8 Changing Mastership of a Replica............................................................... 117 7.9 Deleting a Replica .......................................................................................... 119 8. Managing Mastership ..................................................................................................... 121 8.1 Listing an Object’s Master Replica............................................................... 122 8.2 Listing Objects Mastered by a Replica ........................................................ 123 8.3 Listing the History of Mastership Changes for an Object ........................123 8.4 Displaying Mastership Request Settings .................................................... 124 8.5 Assigning Branch Mastership During Element Creation.........................124 Contents vii /vobs/multisite_doc/manual/ms_manualTOC.fm — September 11, 2001 4:10 pm 8.6 Changing Mastership.....................................................................................126 Transferring Mastership of a Type Object ..................................................127 Transferring Mastership of a Replica Object ..............................................128 Transferring Mastership of a VOB ...............................................................130 Transferring Mastership of an Element.......................................................131 Transferring Mastership of a Branch ...........................................................132 Transferring Branch Mastership ...........................................................132 8.7 Removing Explicit Mastership of a Branch .........................................133 Transferring Mastership of a Stream ...........................................................135 Transferring Mastership of All Objects Mastered by a Replica ...............135 Fixing an Accidental Mastership Change ...................................................137 Working with Type Objects ..........................................................................137 Creating a Shared Type Object .....................................................................137 Listing Whether a Type Object Is Shared or Unshared.............................138 Converting an Unshared Type Object to a Shared Type Object..............138 9. Implementing Requests for Mastership .....................................................................141 9.1 Overview of a Request for Mastership........................................................141 9.2 Requirements and Recommendations.........................................................144 9.3 Planning Your Implementation....................................................................145 To Hide Request for Mastership Features ..................................................145 9.4 Enabling Requests for Mastership ...............................................................146 Prerequisites ....................................................................................................146 Adding Developers to the Access Control List ..........................................146 Deny Requests for Specific Objects ..............................................................148 Enable Requests at the Replica Level...........................................................148 9.5 Customizing Synchronization Updates for Mastership Requests...........149 9.6 Displaying Mastership Request Settings.....................................................150 9.7 Troubleshooting..............................................................................................151 Troubleshooting Commands ........................................................................151 Status Messages ..............................................................................................152 9.8 Serial Development Scenario ........................................................................157 Planning the Implementation .......................................................................157 Setting Up Access Controls ...........................................................................157 viii Administrator’s Guide: Rational ClearCase MultiSite /vobs/multisite_doc/manual/ms_manualTOC.fm — September 11, 2001 4:10 pm Writing Config Specs ..................................................................................... 159 Boston ....................................................................................................... 159 San Francisco ........................................................................................... 159 Tokyo ........................................................................................................ 159 Requesting Mastership .................................................................................. 160 Serial Development of a File That Cannot Be Merged ......................160 Serial Development of a File That Can Be Merged ............................161 10. Troubleshooting MultiSite Operations..................................................................... 163 10.1 Troubleshooting Tips..................................................................................... 164 10.2 Replica-Creation Problems ........................................................................... 165 Export Phase ................................................................................................... 165 Import Phase ................................................................................................... 166 Conflict in VOB Object Registry ........................................................... 166 Conflict in VOB-Tag Registry................................................................ 167 10.3 Synchronization Export Problems ............................................................... 167 Cannot Find Oplog ........................................................................................ 168 Sites Have IP Connection....................................................................... 168 Sites Do Not Have IP Connection......................................................... 169 Oplog Gap Detected During Creation of Update Packet .........................170 Export Failure During Version Construction............................................. 170 Packets Accumulate in Outgoing Storage Bay........................................... 171 Replica Cannot Update Itself........................................................................ 171 10.4 Transport Problems ....................................................................................... 172 Error Messages ............................................................................................... 172 Invalid Destination......................................................................................... 173 Delivery Fails .................................................................................................. 174 Shipping Server Fails to Start or Connection Is Refused..........................174 Shipping Order Expires................................................................................. 175 10.5 Synchronization Import Problems............................................................... 175 Packets Accumulate in Incoming Storage Bay........................................... 176 Packet is Not Applicable to Any Local VOB Replicas ..............................177 Read from Input Stream Fails....................................................................... 178 Contents ix /vobs/multisite_doc/manual/ms_manualTOC.fm — September 11, 2001 4:10 pm Element Changes During Operation ...........................................................178 rmreplica Operation Cannot be Imported ..................................................179 Replica Incarnation is Old .............................................................................180 Miscellaneous Problems ................................................................................181 Recovering from Lost Packets ......................................................................182 Lost Replica-Creation Packet.................................................................182 Lost Update Packet .................................................................................182 Inconsistent Changes to Replica...................................................................184 Ownership Preservation.........................................................................185 Object Mastership....................................................................................186 Automatic Renaming of Type Objects and Replica Objects.....................187 10.6 Running epoch_watchdog ............................................................................188 10.7 Restoring and Replacing Replicas................................................................190 Restoring a Replica from Backup .................................................................191 Replacing an Existing Replica.......................................................................192 Saving Views from the Replaced Replica ............................................195 10.8 Cleaning Up from Accidental Deletion of a Replica .................................196 Using MultiSite for Backup and Interoperability 11. Backing Up VOBs with MultiSite................................................................................199 11.1 Using a Backup Replica .................................................................................199 Handling Objects That Are Not Replicated................................................200 Designing Synchronization Strategy ...........................................................200 11.2 Using Replicas with Incremental Backup ...................................................201 11.3 Restoring a Replica from Backup .................................................................201 12. Using MultiSite for Interoperability ...........................................................................203 12.1 Advantages and Disadvantages ...................................................................203 12.2 Restrictions on Multiple Replicas in a LAN ...............................................204 12.3 Setting Up Multiple Replicas at One Site ....................................................205 x Administrator’s Guide: Rational ClearCase MultiSite /vobs/multisite_doc/manual/ms_manualTOC.fm — September 11, 2001 4:10 pm MultiSite Reference Pages 13. MultiSite Reference Pages .......................................................................................... 209 apropos ................................................................................................................................... 211 chepoch .................................................................................................................................. 213 chmaster ................................................................................................................................. 217 chreplica ................................................................................................................................. 224 epoch_watchdog .................................................................................................................. 227 lsepoch.................................................................................................................................... 229 lsmaster .................................................................................................................................. 232 lspacket................................................................................................................................... 237 lsreplica................................................................................................................................... 240 mkorder................................................................................................................................... 244 mkreplica ................................................................................................................................ 249 MultiSite Control Panel ....................................................................................................... 263 multitool .................................................................................................................................. 268 recoverpacket........................................................................................................................ 272 reqmaster................................................................................................................................ 276 restorereplica ........................................................................................................................ 283 rmreplica................................................................................................................................. 287 shipping.conf......................................................................................................................... 289 shipping_server .................................................................................................................... 294 sync_export_list ................................................................................................................... 297 sync_receive.......................................................................................................................... 305 syncreplica............................................................................................................................. 310 Index ......................................................................................................................................... 321 Contents xi /vobs/multisite_doc/manual/ms_manualTOC.fm — September 11, 2001 4:10 pm xii Administrator’s Guide: Rational ClearCase MultiSite /vobs/multisite_doc/manual/ms_manualTOC.fm — September 11, 2001 4:10 pm Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 Figure 8 Figure 9 Figure 10 Figure 11 Figure 12 Figure 13 Figure 14 Figure 15 Figure 16 Figure 17 Figure 18 Figure 19 Figure 20 Figure 21 Figure 22 Figure 23 Figure 24 Figure 25 Branch Mastership ................................................................................................. 9 Creating an Instance of a Type .......................................................................... 16 Resolving Conflicts in Names of Type Objects ............................................... 22 History of Changes to an Unreplicated VOB................................................... 25 State of a VOB Family ......................................................................................... 25 State of a Replicated VOB ................................................................................... 26 Updates Between Two Replicas......................................................................... 27 Two-Row Epoch Number Matrix at Replica boston_hub..............................28 Epoch Number Matrix at Replica boston_hub ................................................ 30 Unidirectional and Bidirectional Updating ..................................................... 42 One-to-One Synchronization Pattern................................................................ 43 Ring Synchronization Pattern ............................................................................ 43 Single-Hub Synchronization Pattern ................................................................ 44 Multiple-Hub Synchronization Pattern............................................................ 44 Tree Synchronization Pattern............................................................................. 45 Many-to-Many Synchronization Pattern.......................................................... 46 VOB Family Information .................................................................................... 51 VOB Family Feature Levels................................................................................ 80 Replica Synchronization ..................................................................................... 86 Peer-to-Peer Synchronization Pattern............................................................... 87 Hierarchical Synchronization Pattern............................................................... 87 A Synchronization Export Schedule ................................................................. 89 The Store-and-Forward Facility......................................................................... 91 Store-and-Forward Configuration .................................................................... 99 Partial Synchronization Export Pattern and Schedule for Three Replicas .............................................................................................. 110 Figures xiii /vobs/multisite_doc/manual/ms_manualLOF.fm — September 11, 2001 4:03 pm xiv Administrator’s Guide: Rational ClearCase MultiSite /vobs/multisite_doc/manual/ms_manualLOF.fm — September 11, 2001 4:03 pm Tables Table 1 Table 2 Table 3 Table 4 Table 5 Table 6 Table 7 Table 8 Table 9 Table 10 Table 11 Table 12 Table 13 Table 14 Data Propagated Among Replicas ...................................................................... 3 Mastership Restrictions for VOB Objects ......................................................... 17 Disk Space Needed for Storage Bay.................................................................. 34 Choosing a Packet Transfer Method................................................................. 41 multitool Subcommands Copied from ClearCase .......................................... 55 Replica Creation, Synchronization, and Management Commands .............56 Object Mastership Commands........................................................................... 56 Failure-Recovery Commands ............................................................................ 57 Additional MultiSite Commands ...................................................................... 59 ClearCase Commands Related to MultiSite..................................................... 60 Import Methods ................................................................................................. 107 Error Messages from Mastership Request Management Operations ........153 Error Messages from Mastership Requests.................................................... 155 Shipping Error Messages.................................................................................. 172 Tables xv /vobs/multisite_doc/manual/ms_manualLOT.fm — September 11, 2001 4:01 pm xvi Administrator’s Guide: Rational ClearCase MultiSite /vobs/multisite_doc/manual/ms_manualLOT.fm — September 11, 2001 4:01 pm Preface Rational ClearCase MultiSite (abbreviated to “MultiSite” in this manual) is a layered product option to Rational ClearCase. It supports parallel software development and software reuse across project teams distributed geographically and provides automated, error-free replication of versioned object bases (VOBs) and transparent access to all software elements. You can also use MultiSite as an interoperation solution in a mixed UNIX and Windows network, or as a backup mechanism. About This Manual This manual is for all MultiSite administrators. It assumes you have experience with ClearCase. The manual provides an overview of MultiSite, describes how to set up and use it, and gives troubleshooting suggestions. The recommended sequence for reading this manual: Preface ➤ Read Chapter 1 and Chapter 3 of this book for an overview of the product. ➤ Read Chapter 2 to understand MultiSite planning issues. ➤ Read the chapters in the Using MultiSite part of the book. ➤ Read Chapter 11 if you plan to use MultiSite as a backup strategy. ➤ Read Chapter 12 if you plan to use MultiSite for UNIX and Windows interoperation. xvii ClearCase Documentation Roadmap Orientation Introduction Release Notes Online Tutorials Development Project Management Developing Software Managing Software Projects More Information Command Reference Quick Reference Online documentation Build Management OMAKE Guide (Windows platforms) Building Software Administration Installation Guide Administrator’s Guide (Rational ClearCase) Administrator’s Guide (Rational ClearCase MultiSite) Platform Information (See online help) xviii Administrator’s Guide: Rational ClearCase MultiSite Typographical Conventions This manual uses the following typographical conventions: ➤ ccase-home-dir represents the directory into which the ClearCase Product Family has been installed. By default, this directory is /usr/atria on UNIX and C:\Program Files\Rational\ClearCase on Windows. ➤ attache-home-dir represents the directory into which ClearCase Attache has been installed. By default, this directory is C:\Program Files\Rational\Attache, except on Windows 3.x, where it is C:\RATIONAL\ATTACHE. ➤ Bold is used for names the user can enter; for example, all command names, file names, and branch names. ➤ Italic is used for variables, document titles, glossary terms, and emphasis. ➤ A monospaced font is used for examples. Where user input needs to be distinguished from program output, bold is used for user input. ➤ Nonprinting characters are in small caps and appear as follows: , . ➤ Key names and key combinations are capitalized and appear as follows: SHIFT, CTRL+G. ➤ [] Brackets enclose optional items in format and syntax descriptions. ➤ {} Braces enclose a list from which you must choose an item in format and syntax descriptions. ➤ | A vertical bar separates items in a list of choices. ➤ ... In a syntax description, an ellipsis indicates you can repeat the preceding item or line one or more times. Otherwise, it can indicate omitted information. NOTE: In certain contexts, ClearCase recognizes “...” within a pathname as a wildcard, similar to “*” or “?”. See the wildcards_ccase reference page for more information. ➤ If a command or option name has a short form, a “medial dot” ( ⋅ ) character indicates the shortest legal abbreviation. For example: lsc·heckout This means that you can truncate the command name to lsc or any of its intermediate spellings (lsch, lsche, lschec, and so on). Preface xix Online Documentation MultiSite provides a help system that includes an online version of this manual and context-sensitive help for the MultiSite Control Panel and the MultiSite graphical interfaces on Windows. MultiSite provides access to reference pages (detailed descriptions of MultiSite commands, utilities, and data structures) with the multitool man command. The multitool help command displays individual subcommand syntax: multitool help lspacket Usage: lspacket [-long | -short] [pname ...] Technical Support If you have any problems with the software or documentation, please contact Rational Technical Support via telephone, fax, or electronic mail as described below. For information regarding support hours, languages spoken, or other support information, click the Technical Support link on the Rational Web site at www.rational.com. xx Your Location Telephone Facsimile Electronic Mail North America 800-433-5444 toll free or 408-863-4000 Cupertino, CA 408-863-4194 Cupertino, CA 781-676-2460 Lexington, MA support@rational.com Europe, Middle East, and Africa +31-(0)20-4546-200 Netherlands +31-(0)20-4546-201 Netherlands support@europe.rational.com Asia Pacific 61-2-9419-0111 Australia 61-2-9419-0123 Australia support@apac.rational.com Administrator’s Guide: Rational ClearCase MultiSite MultiSite Overview Introduction to MultiSite 1 1 Rational ClearCase MultiSite adds a powerful capability to Rational ClearCase. With MultiSite, developers at different locations can use the same versioned object base (VOB). Each location (site) has a copy (replica) of the VOB. At any time, a site can propagate the changes made in its particular replica to other sites, by sending update packets. The update process can be automatic or can be started manually with a command. An organization can use MultiSite to distribute independent, but related development efforts across multiple cities, nations, or continents. For example, a company in the United States has development and testing sites in India, Argentina, Japan, and Australia. Because it is impractical for all engineers to access the ClearCase VOBs in the United States, the company uses MultiSite to distribute the development. MultiSite can also be used at a single geographical location to allow independent groups to work with the same development data, to enable interoperation in a mixed environment, or to be a backup mechanism. For example, a company that is moving some development to Windows from UNIX can create replicas on Windows instead of accessing UNIX VOBs from Windows. 1.1 VOBs and VOB Replicas In ClearCase, a VOB provides permanent storage for an entire directory tree: directories, files, and links. The historical versions of the files in the VOB are stored in data container files in storage pool directories. The VOB database records the evolution of the version-controlled file-system objects, and stores the associated metadata, including version labels, hyperlinks, configuration records, and so on. For more details on VOB data structures, see the ClearCase documentation set. 1 - Introduction to MultiSite 1 If MultiSite is not used, each VOB has a single set of data containers and a single database. With MultiSite, some or all VOBs are replicated. A replicated VOB is located at multiple sites; at each site is a copy of the VOB, called a VOB replica. Collectively, the set of replicas of a VOB is called a VOB family. Each replica includes a full set of data containers and a complete copy of the VOB database. At its site, a replica appears to be a regular VOB; developers can check out, edit, and check in; build software; attach metadata annotations to objects; and so on. Regular ClearCase use models apply to use of replicas, but there are some site coordination issues that administrators must consider. (For more information, see Chapter 2, Planning a MultiSite Implementation.) Also, MultiSite features support simultaneous development at different replicas without conflicts. Enabling Independent VOB Development: Mastership on page 7 describes how conflict avoidance works. For more details on VOBs and VOB replicas, see VOB Objects and Replica Objects on page 23 and ClearCase Commands Related to MultiSite on page 60. Replica Names, Replica Objects, and Host Assignments Each replica has a replica name in addition to a VOB-tag. You specify both the replica name and the VOB-tag when you create the replica. For each replica, the VOB database contains a replica object that records the name of the replica. The VOB database also tracks the location of each replica by host name. This tracking enables MultiSite administrators to specify replicas at other sites with short, mnemonic identifiers, without needing to know their exact locations. Differences Among Sites Each replica is a copy of the VOB, including both file-system data (data containers) and metadata (VOB database). A developer at any site can see all VOB elements, and all versions of each element. The replicas are not necessarily exact copies of each other. MultiSite features accommodate typical differences among sites: ➤ 2 Different sites may have different user spaces defined by the local password and group databases. You can configure particular replicas to ignore permissions differences, or to propagate changes in permissions from site to site (if the sites support the same user/group database). For more information, see Element Ownership and Ownership Preservation on page 4. Administrator’s Guide: Rational ClearCase MultiSite ➤ Disk configurations and capacities may vary. Accordingly, you can manage VOB storage pools independently at each site. ➤ Different sites may have different development policies and can use site-specific scripts to enforce these policies. For this reason, ClearCase triggers are not propagated among sites. Most, but not all, of the information stored in a VOB is replicated. All changes that create new data, remove old data, and move or rename existing data are propagated among the replicas in the VOB family. Information stored in views is not propagated. For example, a replica update includes the fact that an element has been checked out, because the checkout is recorded in the VOB; but the update does not include the contents of the checked-out version. Table 1 shows the information that is and is not propagated among replicas. Table 1 Data Propagated Among Replicas (Part 1 of 2) Data propagated Data not propagated Elements, branches, versions (including DO versions). Derived objects that have not been checked in as versions. DOs tend to be large and short-lived; transmitting them among multiple sites is likely to be less efficient than rebuilding them at each site. Most kinds of type objects. Trigger type objects. Metadata annotations: version labels, attributes, hyperlinks (including merge arrows and hyperlinks to administrative VOBs). Individual “attached” triggers. ClearCase UCM objects: activities, baselines, components, folders, projects, streams Permanent locks (those created with the –obsolete option). Temporary locks (those created without the –obsolete option). Checkout records of elements and changes in checked-out directories. NOTE: The lscheckout –areplicas command lists checkouts in other replicas. Contents of checked-out versions. 1 - Introduction to MultiSite 3 Table 1 Data Propagated Among Replicas (Part 2 of 2) Data propagated Data not propagated Event records. Mastership information. (See Enabling Independent VOB Development: Mastership on page 7.) Mastership request settings. (See Chapter 9, Implementing Requests for Mastership.) Custom type managers. Changes to text mode property. (When you create a new replica, it has the same text mode property as its parent replica, but subsequent changes are not propagated.) The biggest difference among sites reflects the basic capability of MultiSite: enabling development work to proceed independently at different locations. For more information, see Enabling Independent VOB Development: Mastership on page 7. Element Ownership and Ownership Preservation You can designate some or all replicas in a VOB family to be ownership-preserving. These replicas maintain the same user and group ownerships and permissions on elements, and changes to ownership or permissions are synchronized among them. The ownership of the original VOB is not preserved; the user who enters the mkreplica –import command becomes the owner of the new VOB. Each replica that is not ownership-preserving maintains its own ownership and permissions information for elements. For non-ownership-preserving replicas: 4 ➤ The user who enters the mkreplica –import command becomes the owner of the new VOB and of all elements in it. ➤ This user’s primary group is the group for all elements. ➤ The initial permissions of the elements are the same as their values in the replica at which the mkreplica –export command is entered. Administrator’s Guide: Rational ClearCase MultiSite ➤ Changes to ownership and permissions are not propagated to other replicas. Ownership and permissions changes made at ownership-preserving replicas are ignored at non-ownership-preserving replicas. Requirements for Ownership-Preserving Replicas The sites of ownership-preserving replicas must support the same set of user and group accounts (at least for the accounts that can be assigned to VOB elements). The user and group names and numerical IDs must be the same across sites. For example, on UNIX, the sites must share the same NIS map. On Windows, the replicas must be in the same Windows domain. On UNIX, you can maintain separate but identical user/group databases across domains. On Windows, ownership modes (UIDs and GIDs) are not consistent across domains. Therefore, the entire set of replicas cannot be ownership-preserving in either of the following cases: ➤ All replicas in a VOB family are not in the same Windows domain. ➤ Some replicas in a VOB family are located on UNIX machines, and others are located on Windows machines. You can maintain ownership preservation on a subset of replicas in a VOB family. For example: ➤ A VOB family consists of the replicas bangalore and tokyo, hosted on Windows, and the replicas boston_hub, sanfran_hub, buenosaires, and sydney, hosted on UNIX. The VOB hosts for boston_hub and sanfran_hub are in domains that have the same user/group databases, so boston_hub and sanfran_hub are created as ownership-preserving replicas. ➤ A VOB family consists of five replicas on Windows: seattle, aloha, troy, boston, and boston_backup. All replicas except boston and boston_backup are located in different Windows domains. The replica boston_backup is used as a backup replica for boston, and the hosts for these replicas are in the same Windows domain (but in different ClearCase registry regions). boston and boston_backup are created as ownership-preserving replicas. There can be only one subset of ownership-preserving replicas in a VOB family, even if some replicas do not exchange update packets with all other replicas in the family. NOTE: 1 - Introduction to MultiSite 5 Synchronizing Replicas in a VOB Family Because elements in a replicated VOB are developed concurrently at different sites, the contents of each replica in a VOB family tend to diverge. In fact, a particular replica is rarely—and need never be—in the same state as any other replica. To keep the replicas from diverging too much, each site sends updates to one or more other sites. Updating a replica changes both its database and its storage pools to reflect the development activity that has taken place in one or more other replicas. Replica information is sent in packets. A logical packet includes all the information required to create a new VOB replica (replica-creation packet) or to update one or more existing VOB replicas (update packet). For flexibility, and to accommodate limitations of data-transport facilities, each logical packet can be created as a set of physical packets. After a logical packet is sent to a site, it is processed at that site by a mkreplica or syncreplica command invoked with the –import option. The changes that occurred originally at the sending site (and perhaps some other sites, too) are added to the database and storage pools of the replica at the receiving site. If the logical packet includes several physical packets, the import commands always process the physical packets in the correct order. No error occurs if the same packet is imported two or more times at a site, unless the imports occur simultaneously. You can match the synchronization strategy for each VOB to its particular use patterns, your organization’s needs, and the level of connectivity among the sites. For one VOB, you can update replicas every hour, using a high-speed network; for another VOB, you can send updates only once or twice a month, using electronic mail, magnetic tape (UNIX), or disk files as the delivery mechanism. See MultiSite Use Model on page 38 for information about planning synchronization. Chapter 6, Synchronizing Replicas, discusses the user-level facilities that create and synchronize VOB replicas. VOB Operations and the Oplog on page 24 describes the underlying mechanism that supports the user-level facilities. MultiSite, Time, and Time Zones In ClearCase and MultiSite, time stamps are stored in Universal Coordinated Time (UTC) and are printed to reflect the local time. For example, if a developer in Bangalore checks in a version of a file at 14:33 Bangalore time, the version-creation time is stored as 09:03UTC. When a developer in San Francisco looks at the description of the version, the version-creation time is displayed as 01:03 San Francisco time. 6 Administrator’s Guide: Rational ClearCase MultiSite When you automate synchronization, you must adjust schedules for time zone differences. For an example, see Designing an Update Strategy on page 87. Time rules in config specs are not absolute. The version selected by a time rule can change after an update packet is imported at your site. For example, your config spec has the following time rule, which selects the latest version on the main branch as of July 10 at 7:00 P.M.: element /vobs/dev/plan.txt /main/LATEST –time 10-Jul.19:00 When you put this rule in the config spec, the latest version on the main branch was 17. However, a developer at another site created version 18 on July 10 at 6:00 P.M. your time, but this change has not been propagated to your site. After the update packet that contains the change is imported at your site, your time rule selects version 18. 1.2 Enabling Independent VOB Development: Mastership Because changes to the VOB are made independently at multiple replicas, these changes may conflict. This section describes the mechanism that prevents most conflicts, and Conflict Resolution on page 21 describes the facilities for resolving the unavoidable conflicts. If the development work done in different replicas were truly independent, the result would be chaos. Suppose version 17 of file project.c is created on the main branch in three replicas at the same time. Which is the real version 17, and what ought to happen to the other two versions? An exclusive-right-to-modify scheme, called mastership, avoids such conflicts. Certain VOB-database objects are assigned a master replica (or master). The initial master of an object is the replica where the object is created, and mastership can be changed subsequently (see Chapter 8, Managing Mastership). In general, an object can be modified or deleted only at its master replica. For example, this command fails because it is entered at the boston_hub replica, for a type object mastered by the sanfran_hub replica: SUSHI> cleartool rename lbtype:SF_V2.0 SANFRAN_V2.0 cleartool: Error: Unable to perform operation "rename type" in replica "boston_hub" of VOB "/vobs/dev". cleartool:Error:Master replica of label type "SF_V2.0" is "sanfran_hub". cleartool:Error:Unable to rename type from "SF_V2.0" to "SANFRAN_V2.0". 1 - Introduction to MultiSite 7 Replica Mastership When you create a new replica, its replica object (the VOB database object that records the replica’s existence) is mastered by the creating replica. Therefore, you can modify or delete the replica object only at the creating replica. To facilitate replica maintenance, we recommend that each replica be self-mastering, which means that it is the master of its own replica object. For more information, see Transferring Mastership of a Replica Object on page 128. To perform certain procedures on a replica object, you must make the replica self-mastering. This requirement is documented in those procedures. NOTE: Branch Mastership Branch mastership is the scheme that supports independent development work at different sites. By default, every branch type defined in a VOB (including the main branch type) is mastered by one replica in a VOB family. By default, branches can be created and modified only at the replica that masters the branch type. Checking out a version is considered a branch modification. (The exception to the creation rule is the creation of the main branch; for more information, see Creation of the main Branch of an Element on page 10.) Remember that a branch is an instance of a branch type. For example, main is a branch type, and acc.c@@/main and resource.h@@\main are branches. NOTE: The branch mastership strategy works with the standard ClearCase strategy of using branches to isolate changes for particular development tasks. (For example, fixing a defect may require changes to 5 elements, where each change is made on a branch named v1.0_bugfix.) With MultiSite, work on various tasks can be done at different sites, each using its own branch. The work on different branches can be propagated among sites, and then merged, as often as required by an organization’s development strategy. Because the branches of an element are independent, changes made at different sites do not conflict. Figure 1 illustrates branch mastership strategy: each replica masters a branch type and can create versions only on the branch of that type. 8 Administrator’s Guide: Rational ClearCase MultiSite Figure 1 Branch Mastership boston_hub main main main v1.0_integration v1.0_bugfix bangalore sanfran_hub v1.0_integration v1.0_bugfix v1.0_integration v1.0_bugfix Branch mastership is implemented at both the branch type level and the branch level: ➤ By default, the replica in which a branch type is created masters the branch type and all instances of that branch type. For example, the sanfran_hub replica masters the branch type object named v1.0_integration and owns the right to modify v1.0_integration branches in all of the elements in the VOB. ➤ An administrator or developer can transfer the mastership of an individual branch (an instance of a branch type) to another replica. This feature enables serial development. For example, if a developer at the Boston site needs to work on the v1.0_integration branch for the element main.c, the San Francisco administrator can transfer mastership of the branch main.c@@/main/v1.0_integration to boston_hub, or the developer can request mastership of the branch. For more information on supporting serial development with MultiSite, see Supporting Serial Development in Replicas on page 20. 1 - Introduction to MultiSite 9 Creation of the main Branch of an Element There is an exception to the rule that a branch can be created only at the master replica of the branch type. When you add a file to source control or create a new directory element, the main branch is created even if your current replica does not master the main branch type. By default, the main branch of a new element is mastered by the replica that masters the main branch type, and you cannot create new versions on the branch. During element creation, you can specify an option to have your current replica master all newly created branches. For more information, see Assigning Branch Mastership During Element Creation on page 124. Synchronizing Development on Different Branches Development of an element with multiple branches can take place in different replicas concurrently, with occasional synchronizations. (The more frequently you update, the easier it is to track and reconcile the changes on different branches of elements. To reconcile changes, you use the ClearCase version-comparison and merge facilities.) For example, before the Boston site starts using MultiSite, the element cmdsyn.c has two branches, main and v1.0_integration: main v1.0_integration When the Boston site starts using MultiSite, the administrator creates a new replica for the San Francisco site. Because integration for Version 1.0 will be done at the San Francisco site, the sanfran_hub replica must master the v1.0_integration branch type. The administrator transfers mastership of the v1.0_integration branch type to the sanfran_hub replica. Developers in San Francisco can now create versions on the v1.0_integration branch of cmdsyn.c and can create instances of the v1.0_integration branch type for other elements. Work continues on the main branch in Boston: 10 Administrator’s Guide: Rational ClearCase MultiSite boston_hub sanfran_hub main main v1.0_integration v1.0_integration The administrators at the Boston and San Francisco sites decide to merge some of the work on the v1.0_integration branch with the work done on the main branch. The San Francisco administrator sends an update packet to the boston_hub replica, and the Boston administrator imports it: 1 - Introduction to MultiSite 11 boston_hub sanfran_hub main main v1.0_integration v1.0_integration The Boston administrator then merges from the v1.0_integration branch to the main branch by checking out the latest version on the main branch, merging from the latest version on the v1.0_integration branch, and checking in the result of the merge: 12 Administrator’s Guide: Rational ClearCase MultiSite Default and Explicit Branch Mastership Branches can have default mastership or explicit mastership. When a branch is created, it is mastered by the replica that masters the branch type (default mastership). When you transfer mastership of a branch to another replica, that replica masters the branch explicitly. The output of describe shows how a branch is mastered. For example, the branch type v2.0_port was created at, and is mastered by, the sanfran_hub replica. The test2.txt@@/main/v2.0_port branch has default mastership, as shown by the (defaulted) annotation: multitool describe test2.txt@@/main/v2.0_port branch “test2.txt@@/main/v2.0_port” created 18-Aug-00.10:50:34 by John Cole (jcole.user@goldengate) branch type: v2.0_port master replica: sanfran_hub@/vobs/dev (defaulted) ... The administrator at the sanfran_hub replica transfers mastership of this branch to the boston_hub replica: 1 - Introduction to MultiSite 13 multitool chmaster –nc boston_hub test2.txt@@/main/v2.0_port Changed mastership of branch "/vobs/dev/test2.txt@@/main/v2.0_port" to "boston_hub" The output of describe shows that this branch is now mastered explicitly by the boston_hub replica; the (defaulted) annotation is not present: multitool describe test2.txt@@/main/v2.0_port branch “test2.txt@@/main/v2.0_port” created 18-Aug-00.10:50:34 by John Cole (jcole.user@goldengate) branch type: v2.0_port master replica: boston_hub@/vobs/dev ... When you transfer mastership of a branch type, mastership is transferred for all branches of that type with default mastership. Mastership of branches with explicit mastership is not transferred. For more information, see the chmaster reference page and Transferring Mastership of a Branch on page 132. Type Object Mastership By default, you can create an instance of a type only in the replica that masters the type object. For example, if the sanfran_hub replica masters the TESTED_BY attribute type, you can create a TESTED_BY attribute only in the sanfran_hub replica. Often, however, developers at different sites must create instances of the same type. For example, quality engineers at the bangalore replica may also use the TESTED_BY attribute. Therefore, the mkattype, mkhltype, and mklbtype commands can create two kinds of type objects: ➤ Instances of an unshared type object can be created only in its master replica. (The instances are propagated to and seen in all replicas.) Thus, there are no issues with conflicting changes made in different replicas. By default, the mkattype, mkhltype, and mklbtype commands create unshared type objects. ➤ Instances of a shared type object can be created in multiple replicas. To prevent cross-replica conflicts, the following restrictions apply: ➣ 14 The current replica must master the target object (the object to which the annotation is being attached). Administrator’s Guide: Rational ClearCase MultiSite ➣ The ClearCase-level instance restrictions (if any) on the type object must allow creation of the instance. NOTE: If a hyperlink type is shared, you can create a hyperlink of that type between any two objects, at any replica. ClearCase restrictions that prevent instance creation in an unreplicated VOB also prevent instance creation in a replica; for example, if there is a lock on the type object, instance creation fails. However, because locks are not replicated (except for locks created with –obsolete), a lock on a shared type object in one replica does not prevent instance creation in another replica. An unshared type object can be converted to shared, but a shared type cannot be converted to unshared. Instance restrictions (for example, once-per-branch use of a label type) for a shared type object cannot be changed. Figure 2 illustrates the restrictions on creating an instance of a shared type object. For all target objects except versions and branches, the current replica must master the target object. This is slightly different for versions and branches: ➤ For a version, the current replica must master the branch on which the version is located. When you apply a label whose instance restriction is one per branch, your current replica must master the branch. When you apply a label whose instance restriction is one per element, your current replica must master the element. NOTE: ➤ For a branch with default mastership, the current replica must master the branch type. ➤ For a branch with explicit mastership, the current replica must master the branch object. 1 - Introduction to MultiSite 15 Figure 2 Creating an Instance of a Type Can I create an instance? Is the type shared? No Do you master the type? No Yes Do you master the target object? Yes Yes No You can create an instance (if no ClearCase-level restrictions exist). You cannot create an instance. You can create an instance (if no ClearCase-level restrictions exist). You cannot create an instance. For example, the administrator at boston_hub creates an attribute type with the following command: cleartool mkattype –shared –vpbranch –nc TESTED This attribute type is defined to be shared across replicas, with the restriction that at most one instance can be created on each branch of an element. You can create an attribute of that type on a version if both of the following are true: 16 ➤ Your current replica masters that version’s branch. ➤ No attribute of that type already exists on a version on that branch (assuming no other ClearCase restrictions). Administrator’s Guide: Rational ClearCase MultiSite Additional mastership restrictions exist when you use administrative VOBs and global types. If a global type is shared, ClearCase can create a local copy of the type only if the type is mastered by the administrative VOB replica at the current site. If the shared global type is not mastered at the current site, you can create instances of the type only if the client VOB replica contains a local copy of the type. This restriction applies even if your current replica masters the object to which you are attaching the instance. This mastership restriction prevents conflicting, simultaneous creation of a given type with a given name at multiple sites. For more information, see Administrator’s Guide for Rational ClearCase. For more information about changing type mastership, see Chapter 8, Managing Mastership. Mastership Restrictions Table 2 describes the restrictions for VOB objects. Table 2 Mastership Restrictions for VOB Objects Object Action Object your current replica must master Activity Change (chactivity) Remove (rmactivity) Set (setactivity) Activity Attribute Create (mkattr) Type (if the attribute’s type is unshared) Object to which attribute is being applied (if the attribute’s type is shared) Remove (rmattr) Type (if the attribute’s type is unshared) Object from which attribute is being removed (if the attribute’s type is shared) Create (mkbl) Stream where you make the baseline. For an imported baseline created from a pre-UCM label, your current replica must master the component and label type. Label (mklabel) Stream’s branch type (in each VOB where you have made changes) Change (chbl) Remove (rmbl) Baseline Baseline 1 - Introduction to MultiSite 17 Table 2 Mastership Restrictions for VOB Objects Object Action Object your current replica must master Branch Change type (chtype) New branch type and the branch you are changing Create (mkbranch) Branch type Remove (rmbranch) Branch Checked-out version Reserve (reserve) Branch on which the version is checked out Component Remove (rmcomp) Component Element Check in (checkin) Branch on which you are checking in the version Check out (checkout) Branch on which you are checking out the version (unless you use –unreserved –nmaster) Change type (chtype) Relocate (relocate) Remove (rmelem) Element Event record Change (chevent) For a version, the branch containing the version. For any other object, the object. Folder Change (chfolder) Remove (rmfolder) Folder Hyperlink Create (mkhlink) Hyperlink type (for unshared types) Remove (rmhlink) Hyperlink Create (mklabel) Remove (rmlabel) If the label’s type is unshared, your current replica must master the label type. If the label’s type is shared, the following restrictions apply: Label Merge arrow 18 Remove (rmmerge) ➤ If the label type is one per branch, your current replica must master the branch containing the version. ➤ If the label type is one per element, your current replica must master the version’s element. Merge hyperlink Administrator’s Guide: Rational ClearCase MultiSite Table 2 Mastership Restrictions for VOB Objects Object Action Object your current replica must master Object Change event (chevent) Change mastership (chmaster) Change name (rename) Lock obsolete (lock –obsolete) Unlock (unlock) Object Change protection (protect) Object (if current replica is ownership-preserving) Project Change (chproject) Remove (rmproject) Project Project VOB Change list of promotion levels (setplevel) PromotionLevel attribute type Replica Change host (chreplica) Change ownership-preservation properties (chreplica) Enable requests for mastership (reqmaster) Remove (rmreplica) Replica Stream Change (chstream) Rebase (rebase) Remove (rmstream) Stream Symbolic link Remove (rmelem) Symbolic link Type Copy (cptype) The replica containing the original type must master that type. Remove (rmtype) Replace (mk**type –replace) Type Check in (checkin) Check out (checkout) Remove (rmver) Branch With checkout –unreserved –nmaster, there are no mastership restrictions. Version 1 - Introduction to MultiSite 19 Mastership Restrictions for VOB Objects Table 2 Object Action Object your current replica must master VOB Change feature level (chflevel) The replica to be changed must be self-mastering. Change protection (protectvob) VOB (for ownership-preserving replicas) Set up snapshots (vob_snapshot_setup) The replica must be self-mastering. Change feature level (chflevel) VOB object VOB family 1.3 Supporting Serial Development in Replicas The standard ClearCase development model is to use branches to develop software in parallel, and the standard MultiSite model is to master different branch types at different replicas. These models require you to merge changes from branch to branch. However, sometimes sites must use serial development (for example, to make changes to elements whose versions cannot be merged). To support serial development, there are two models for changing mastership: ➤ Push Model The developer who needs to work on a branch asks the administrator at the master replica’s site to transfer mastership of the branch and send an update packet containing the change. ➤ Pull Model The developer who needs to work on a branch requests mastership of the branch. This model is not enabled by default, and it requires the MultiSite administrator to enable requests and authorize developers to request mastership. However, after the setup is complete, the administrator does not need to be involved in the mastership request process. NOTE: The developer can also request mastership of branch types. For more information, see Chapter 9, Implementing Requests for Mastership. 20 Administrator’s Guide: Rational ClearCase MultiSite There are two ways to use requests for mastership: ➣ If you cannot merge versions of the element, you must request mastership, and after your current replica receives mastership, you can perform a reserved checkout and do your work. ➣ If you can merge versions of the element, you can perform a nonmastered checkout of the element and do your work. At any time, request mastership. When your current replica receives mastership, merge your work (if required) and check in the file. For more information about enabling requests for branch mastership, see Chapter 9, Implementing Requests for Mastership. For more information about the use models for requesting mastership, see Working On a Team in Developing Software. 1.4 Conflict Resolution Mastership restrictions prevent most inconsistent changes in different VOB replicas, but some inconsistent changes are unavoidable. For example, a label type named V3.0 can be created at two or more replicas at the same time. (The actual times can be quite different: between updates, while replicas evolve independently, a label type creation operation in one replica is logically simultaneous with all label type creations in the other replicas.) To avoid many naming conflicts, the ClearCase and MultiSite administrators for a VOB family must create and enforce some naming and use rules for objects in VOBs. A ClearCase use model that is used consistently across sites reduces the potential for conflicts. For example, the administrators for a VOB family agree that all site-specific labels must include a site identifier, and labels that will be used at multiple sites are created only at a certain site. Resolving Conflicts Among Type Objects Two objects of the same type in the same VOB cannot have identical names. Accordingly, the syncreplica –import command detects a conflict when an update packet includes an operation that would create a type object with the same name as an existing object at the current replica. It resolves the conflict by creating the new type object with a different name. For example, in Figure 3, two types created at two different replicas have the same name but are different objects. When the type created at the boston_hub replica is imported at the bangalore replica, it is not renamed because the bangalore replica does not contain a type with that name. 1 - Introduction to MultiSite 21 However, when the type created at the sanfran_hub replica is imported at the bangalore replica, it is renamed because the bangalore replica already has a type with that name. Figure 3 Resolving Conflicts in Names of Type Objects replica: boston_hub replica: sanfran_hub V3.0 V3.0 replica: bangalore V3.0 sanfran_hub:V3.0 syncreplica generates a warning message when it renames an object during import. To resolve the conflict, the Bangalore administrator must inform the Boston and San Francisco administrators of the name conflict, and they must take one of the following actions: ➤ Rename both label types. For example, at Boston: multitool rename lbtype:V2.0 V2.0_boston_hub At San Francisco: multitool rename lbtype:V2.0 V2.0_sanfran_hub The Boston and San Francisco administrators must then send updates to the bangalore replica. ➤ Rename one of the label types. The administrator who renames the label type sends an update to the other replicas. For more information, see Automatic Renaming of Type Objects and Replica Objects on page 187. 22 Administrator’s Guide: Rational ClearCase MultiSite 1.5 VOB Objects and Replica Objects It is useful to distinguish these two kinds of objects in the VOB database: ➤ VOB object. The database has a single VOB object. This object’s UUID is listed as the VOB family uuid in a lsvob –long listing. ➤ VOB-replica object. The database has a VOB-replica object for each of the VOB’s replicas. This object’s UUID is listed as the VOB replica uuid in a lsvob –long listing. For example: cleartool lsvob –long /vobs/dev VOB family UUID VOB replica UUID Tag: /vobs/dev Global path: /net/minuteman/vobstg/dev.vbs Server host: minuteman Access: public Mount options: Region: purpledoc_unix Active: YES Vob tag replica uuid: 87f6265f.72d911d4.a5cd.00:01:80:c0:4b:e7 Vob on host: minuteman Vob server access path: /vobstg/dev.vbs Vob family uuid: 87f6265b.72d911d4.a5cd.00:01:80:c0:4b:e7 Vob replica uuid: 87f6265f.72d911d4.a5cd.00:01:80:c0:4b:e7 Use describe vob: to list details about the VOB object; use describe replica: to list details about the VOB-replica object (the replica). All replicas of a VOB record the same VOB object and set of VOB-replica objects. (When a new replica is created, it takes some time for the change—creation of a new VOB-replica object—to be propagated to all the replica’s databases.) 1 - Introduction to MultiSite 23 1.6 VOB Operations and the Oplog This section describes the VOB database mechanism that supports replica synchronization. This information is not required to use MultiSite, but is helpful when you want to deepen your understanding of the error-recovery facilities described in Chapter 10, Troubleshooting MultiSite Operations. Most changes made to a VOB are recorded as event records in the VOB database. Each event record describes a change. For a replicated VOB, operation log entries (oplogs) are created also. These entries store all the information required to replay the changes in another replica: ➤ The identity of the replica where the change originally took place. ➤ The change to the VOB database; for example, creation of a new element, checkin of a new version, attaching of an attribute, and so on. ➤ The change to the storage pool, if any; for example, the contents of a new version. Version information is not stored in the oplog. When version information is required by syncreplica, it is retrieved from the pools. NOTE: ➤ The event record generated for the change. ➤ An integer sequence number: 1 for the first change originating at a particular replica, 2 for the next change, and so on. This is called the epoch number or oplog-ID of the oplog entry. The exact kind and amount of information varies with the specific operation. For example, an oplog entry for the removal of a label has different, and less, information than an oplog entry for a checkout command. NOTE: Oplog entries are created only for replicated VOBs. You can scrub a replica’s oplog entries after they have been used to update other replicas. For more information, see Scrubbing Parameters for VOB Replicas on page 47. 24 Administrator’s Guide: Rational ClearCase MultiSite Tracking Operations for Each Replica The history of an unreplicated VOB is a linear sequence of operations (Figure 4). time Figure 4 History of Changes to an Unreplicated VOB operation 3 operation 2 operation 1 changes to database For a replicated VOB, changes are tracked separately for each replica. (That is why an oplog entry includes the identity of the replica where the operation originated.) Thus, the history of a replicated VOB can be viewed as several stacks of oplog entries. Each stack is represented by a linear sequence of epoch numbers for the operations originating in that replica. Figure 5 shows the state of two replicas in a VOB family: ➤ Operations with epoch numbers 1–950 have occurred at replica boston_hub. ➤ Operations 1–702 have occurred at replica sanfran_hub. Figure 5 State of a VOB Family 950 702 001 boston_hub 1 - Introduction to MultiSite 001 sanfran_hub 25 A replica has accurate data only about its own operations. Until it receives update packets, its information about other replicas is out of date. For example, replica boston_hub records 950 local operations, but has received update packets for only 504 sanfran_hub operations. Similarly, replica sanfran_hub records 702 local operations, but has no current data about the boston_hub replica’s state. Figure 6 illustrates this scenario, in which each replica is out of date with respect to the operations originating at the other replica. Figure 6 State of a Replicated VOB boston_hub replica sanfran_hub replica 950 791 702 504 001 boston_hub 001 sanfran_hub 001 boston_hub 001 sanfran_hub Epoch Numbers Picturing a replicated VOB as a set of oplog stacks, shown in Figure 6, makes it easy to understand the synchronization process. For example, an update packet sent from replica boston_hub to replica sanfran_hub consists of increments to the stack for replica boston_hub (operations 792–950). Figure 7 shows the two increments. Because sanfran_hub knows its own state, it needs updates only for other replicas. (In certain error-recovery situations, you must reset a replica’s data about its own operations. See Chapter 10, Troubleshooting MultiSite Operations.) 26 Administrator’s Guide: Rational ClearCase MultiSite Updates Between Two Replicas Figure 7 boston_hub replica sanfran_hub replica update received from boston_hub 950 update received from sanfran_hub 702 505 950 792 791 702 001 001 504 001 boston_hub 001 sanfran_hub boston_hub sanfran_hub By the time the packet is imported at sanfran_hub, additional VOB-level changes may have been made at boston_hub. These changes are not included in the update packet. NOTE: Optimization and the Epoch Number Matrix The MultiSite synchronization scheme attempts to minimize the amount of data transmitted among sites. Each replica keeps track of these epoch numbers: 1. Changes made in the current replica. The epoch number that indicates how many operations originated at the current replica. 2. Changes at sibling replicas. When syncreplica writes an operation from an update packet to the current replica, it increments the epoch number for the sibling replica at which the operation originally occurred. This epoch number is the number of operations originating at the sibling replica that have been imported at the current replica. 3. Current knowledge of the states of other replicas. For each other replica, an estimate of its own changes and other replicas’ changes. Figure 8 shows how these epoch numbers fall into an epoch number matrix. Each replica maintains its own such matrix, revising its rows as work occurs locally and as it exchanges update packets with other replicas: 1 - Introduction to MultiSite 27 ➤ When work occurs in the boston_hub replica, its own number of oplog IDs is incremented. ➤ When the boston_hub replica generates an update packet to be sent to sanfran_hub, it revises the sanfran_hub row in its epoch number matrix. Note that a syncreplica –export command updates epoch numbers immediately. It does not wait for acknowledgment from the receiving site that the packet has been received and applied correctly. During normal ClearCase and MultiSite processing, no manual intervention is required to maintain the accuracy of the epoch number matrices for the various replicas. However, failure to apply a packet may require manual intervention, as described in Lost Update Packet on page 182. ➤ When the boston_hub replica receives an update from sanfran_hub, it revises its own row (boston_hub) and the sanfran_hub row in its epoch number matrix. Figure 8 Two-Row Epoch Number Matrix at Replica boston_hub Operations originated at boston_hub Operations originated at sanfran_hub boston_hub s record of its own state 950 504 boston_hub s estimate of sanfran_hub s state 912 504 The contents of this matrix are reported by the multitool lsepoch command at the boston_hub replica: multitool lsepoch For VOB replica "/vobs/dev": Oplog IDs for row "boston_hub" (@ minuteman): oid:87f6265f.72d911d4.a5cd.00:01:80:c0:4b:e7=950 oid:0eaa6fc3.737d11d4.adbe.00:01:80:c0:4b:e7=504 Oplog IDs for row "sanfran_hub" (@ goldengate): oid:87f6265f.72d911d4.a5cd.00:01:80:c0:4b:e7=912 oid:0eaa6fc3.737d11d4.adbe.00:01:80:c0:4b:e7=504 (boston_hub) (sanfran_hub) (boston_hub) (sanfran_hub) A syncreplica –export command entered at boston_hub uses this matrix as follows to generate an update destined for sanfran_hub: 28 Administrator’s Guide: Rational ClearCase MultiSite 1. At boston_hub, the number of local operations is 950 (number in upper-left corner of matrix), and the estimate is that sanfran_hub has been updated only to epoch number 912 (number in lower-left corner). 2. The update packet that boston_hub sends to sanfran_hub includes boston_hub oplog entries 913-950. After the Boston administrator invokes syncreplica –export, the sanfran_hub row is updated: multitool lsepoch For VOB replica "/vobs/dev": Oplog IDs for row "boston_hub" (@ minuteman): oid:87f6265f.72d911d4.a5cd.00:01:80:c0:4b:e7=950 oid:0eaa6fc3.737d11d4.adbe.00:01:80:c0:4b:e7=504 Oplog IDs for row "sanfran_hub" (@ goldengate): oid:87f6265f.72d911d4.a5cd.00:01:80:c0:4b:e7=950 oid:0eaa6fc3.737d11d4.adbe.00:01:80:c0:4b:e7=504 (boston_hub) (sanfran_hub) (boston_hub) (sanfran_hub) Indirect Synchronization If there are more than two replicas in a VOB family, synchronization can occur indirectly. A replica can include nonlocal changes in update packets. For example, if boston_hub exchanges updates with replicas sanfran_hub and bangalore, it sends bangalore oplog entries that it has received previously from sanfran_hub. These entries may or may not bring replica bangalore up to date on sanfran_hub’s changes. (An update sent from sanfran_hub to bangalore does bring bangalore up to date.) NOTE: If a replica does not receive packets directly from some replicas in the VOB family, its rows for those replicas may contain zeros. This is expected behavior. Figure 9 shows replica boston_hub’s epoch number matrix. 1 - Introduction to MultiSite 29 Figure 9 Epoch Number Matrix at Replica boston_hub Operations originated at boston_hub Operations originated at sanfran_hub Operations originated at bangalore boston_hub s record of its own state 950 504 653 boston_hub s record of sanfran_hub s state 912 504 653 boston_hub s record of bangalore s state 709 221 653 The contents of this matrix are reported by the lsepoch command: multitool lsepoch For VOB replica "/vobs/dev": Oplog IDs for row "boston_hub" (@ minuteman): oid:87f6265f.72d911d4.a5cd.00:01:80:c0:4b:e7=950 oid:7ag3b0bc.defa11d0.ba57.00:01:72:73:3c:94=653 oid:0eaa6fc3.737d11d4.adbe.00:01:80:c0:4b:e7=504 Oplog IDs for row "bangalore" (@ ramohalli): oid:87f6265f.72d911d4.a5cd.00:01:80:c0:4b:e7=709 oid:7ag3b0bc.defa11d0.ba57.00:01:72:73:3c:94=653 oid:0eaa6fc3.737d11d4.adbe.00:01:80:c0:4b:e7=221 Oplog IDs for row "sanfran_hub" (@ goldengate): oid:87f6265f.72d911d4.a5cd.00:01:80:c0:4b:e7=912 oid:7ag3b0bc.defa11d0.ba57.00:01:72:73:3c:94=653 oid:0eaa6fc3.737d11d4.adbe.00:01:80:c0:4b:e7=504 (boston_hub) (bangalore) (sanfran_hub) (boston_hub) (bangalore) (sanfran_hub) (boston_hub) (bangalore) (sanfran_hub) A syncreplica –export command at Boston uses this matrix to export an update for bangalore: 1. At Boston, there are 950 local operations (number in upper-left corner of matrix), and the estimate is that bangalore has been updated only to epoch number 709 (lower-left corner). 2. For operations that originated at sanfran_hub, boston_hub has been updated to epoch number 504, and estimates that bangalore has been updated only to epoch number 221. 3. The update packet that boston_hub sends to bangalore includes boston_hub oplogs 710-950 and sanfran_hub oplogs 222-504. The output of a multitool lsepoch command at Boston now looks like this: 30 Administrator’s Guide: Rational ClearCase MultiSite multitool lsepoch For VOB replica "/vobs/dev": Oplog IDs for row "boston_hub" (@ minuteman): oid:87f6265f.72d911d4.a5cd.00:01:80:c0:4b:e7=950 oid:7ag3b0bc.defa11d0.ba57.00:01:72:73:3c:94=653 oid:0eaa6fc3.737d11d4.adbe.00:01:80:c0:4b:e7=504 Oplog IDs for row "bangalore" (@ sushi): oid:87f6265f.72d911d4.a5cd.00:01:80:c0:4b:e7=950 oid:7ag3b0bc.defa11d0.ba57.00:01:72:73:3c:94=653 oid:0eaa6fc3.737d11d4.adbe.00:01:80:c0:4b:e7=504 Oplog IDs for row "sanfran_hub" (@ goldengate): oid:87f6265f.72d911d4.a5cd.00:01:80:c0:4b:e7=912 oid:7ag3b0bc.defa11d0.ba57.00:01:72:73:3c:94=653 oid:0eaa6fc3.737d11d4.adbe.00:01:80:c0:4b:e7=504 1 - Introduction to MultiSite (boston_hub) (bangalore) (sanfran_hub) (boston_hub) (bangalore) (sanfran_hub) (boston_hub) (bangalore) (sanfran_hub) 31 32 Administrator’s Guide: Rational ClearCase MultiSite Planning a MultiSite Implementation 2 2 Before you install and use Rational ClearCase MultiSite, you need to plan your implementation. The plan should include the following items: ➤ MultiSite installation ➤ MultiSite licensing ➤ ClearCase use model ➤ MultiSite use model ➤ Responsibilities of MultiSite administrators This chapter describes these issues in more detail. We recommend that you document your plan in writing and implement your design decisions in a set of test replicas before changing your development environment. 2 - Planning a MultiSite Implementation 33 2.1 MultiSite Installation For MultiSite installation instructions, see the Installation Guide for the ClearCase Product Family. You must install MultiSite on all VOB server hosts where replicated VOBs will reside; replica creation and synchronization imports must occur on the host where the replica resides. You do not need to install MultiSite on your computer to manage mastership, because the MultiSite object mastership commands are available in cleartool. However, you may want to install MultiSite on your computer so you have convenient access to other MultiSite commands. You do not need to install MultiSite on ClearCase client hosts or on server hosts that will not host replicated VOBs. Each VOB server host where replicas will reside must have enough disk space for the MultiSite storage bay directories. The storage bays hold MultiSite packets, along with their corresponding shipping order files. Table 3 describes the amount of available disk space needed on the disk partition where the storage bay is located. Table 3 Disk Space Needed for Storage Bay Type of packet Disk space needed replica-creation Size of VOB database and VOB source pools. update On Windows, twice the size of the largest packet to be stored in bay. The reason is that there may be two instances of the same packet in the bay at one time: one on its way to another destination, and another waiting to be applied to the replica on the current host. On UNIX, the size of largest packet to be stored in bay. There is no specific formula for determining how large your update packets will be. The general rule is that more frequent synchronization results in smaller packets. However, even if you synchronize every hour, a large amount of development activity or release activity can occur in an hour (for example, all of the executables for a release are checked in just before a build, or the release engineer labels all the versions for a release) and can cause a large packet to be generated. If you are not sure that the available disk space can accommodate an unexpectedly large packet, you can configure MultiSite to limit the size of an update packet. For more information on specifying storage bays, see the shipping.conf (UNIX) and MultiSite Control Panel (Windows) reference pages. 34 Administrator’s Guide: Rational ClearCase MultiSite 2.2 MultiSite Licensing A MultiSite license is required for any access to an object in a replicated VOB—by a MultiSite command, a ClearCase command, or a standard operating system command. You can calculate the number of MultiSite licenses your site needs by determining how many developers will access replicated VOBs. If all of your developers will access replicated VOBs, you need the same number of MultiSite licenses as ClearCase licenses. If not all developers will access replicated VOBs, you can purchase fewer MultiSite licenses. For example, a company has two sites, with 20 developers at site A and 5 developers at site B. The company has three VOBs at site A; two of them will be replicated to site B and one will not be replicated. Five of the developers at site A will access only the unreplicated VOB, and the remaining 15 will work in all VOBs. Therefore, the company needs to purchase the following numbers of licenses: Site Number of ClearCase licenses Number of MultiSite licenses A 20 15 B 5 5 This example assumes that you purchase a ClearCase license for each developer. If you have fewer ClearCase licenses than developers, you can purchase a proportionate number of MultiSite licenses. For example, if site B purchased three ClearCase licenses, they would also purchase three MultiSite licenses. NOTE: For more information about MultiSite license allocation and purchasing licenses, see the Installation Guide for the ClearCase Product Family. 2.3 ClearCase Use Model Before development work is started in any VOB, the project manager and administrator must define the ClearCase use model. For example, the project manager must specify the branches, labels, and triggers that are used for development and integration work. The following sections describe the ways in which MultiSite use affects this planning. 2 - Planning a MultiSite Implementation 35 Branching and Mastership Mastership restrictions affect the choices you make about branching and merging: ➤ A common branching strategy is to use a single release branch (or integration branch) and multiple development branches. The project manager or developer merges changes from the development branch to the integration branch. You can use this strategy with MultiSite, but the merges to the integration branch must occur at the replica that masters the integration branch. You could also use a single release integration branch, multiple site integration branches, and multiple developer branches. With this scenario, developers or project managers at a replica merge to the site integration branch, and the project manager at the replica that masters the release integration branch merges to that branch from the site integration branches. You may need to allow developers to transfer and request mastership of branches and branch types. Developers at different sites may have to use the same branch type (for example, because an element’s versions can’t be merged, or because each site must merge its own work to the integration branch). A branch or branch type’s mastership cannot be shared by multiple replicas; instead, there are two models for transferring mastership between sites: Model 1. Create a schedule that determines when each site masters the branch or branch type. Create scripts to transfer mastership. Model 2. Give the developers at the sites the ability to request mastership of the branch or branch type. For more information about this model, see Chapter 9, Implementing Requests for Mastership. NOTE: Do not use mastership transfer models as substitutes for good branching and merging rules. Enabling requests for mastership involves more planning and setup than implementing a strategy for branching and merging. Also, if you can develop in parallel, planned branching and merging is safer than allowing developers to request mastership and merge their own work randomly. ➤ You can use auto-make-branch rules in config specs only if the current replica masters the branch type in the rule. For example, if your current replica masters the v1.0_bugfix branch type but not the v1.0 branch type, this config spec is incorrect because the v1.0 branch cannot be created at this replica: element element element element 36 * * * * CHECKEDOUT .../v1.0_bugfix/LATEST .../v1.0/LATEST -mkbranch v1.0_bugfix /main/LATEST -mkbranch v1.0 Administrator’s Guide: Rational ClearCase MultiSite ➤ By default, when you create an element in a replicated VOB, mastership of the branch main is assigned to the replica that masters the branch type main. If this replica is not your current replica, you cannot create new versions on the main branch. Also, if your config spec contains mkbranch rules and your current replica does not master the branch types, the branches cannot be created during element creation. You can assign mastership of a new element’s main branch and other branches created during element creation to your current replica. For more information, see Assigning Branch Mastership During Element Creation on page 124. Use of Metadata Mastership restrictions affect the way you use ClearCase attributes, labels, or hyperlinks. You need to decide whether these types must be shared. You can create instances of an unshared type only in the replica that masters it. You can create instances of a shared type only in the replica that masters the object to which you are attaching the instance. For more information, see Type Object Mastership on page 14. Trigger types and triggers are not replicated. If a trigger is in use at one replica and needs to be used at other replicas, you must send the appropriate information (for example, the output of a describe trtype: command and the contents of any associated scripts) to the administrators at the other sites. Text Mode for Replicas When you create a new replica, it has the same text mode as the replica from which it was exported. However, changes to a replica’s text mode are not propagated to the other replicas in the family, so if you make a text mode change that needs to occur at all replicas in the family, you and the other MultiSite administrators must change the text mode at each replica. For more information about text modes, see the Administrator’s Guide for Rational ClearCase. 2 - Planning a MultiSite Implementation 37 Use of Administrative VOBs or UCM If replicated VOBs use global types, the administrative VOBs must be replicated. For more information on global types, see the Administrator’s Guide for Rational ClearCase. NOTE: If a global type is shared, Rational ClearCase can create a local copy of the type only if the type is mastered by the administrative VOB replica at the current site. If the shared global type is not mastered at the current site, you can create instances of the type only if the client VOB replica contains a local copy of the type. This restriction applies even if your current replica masters the object to which you are attaching the instance. This mastership restriction prevents conflicting, simultaneous creation of a given type with a given name at multiple sites. For more information, see Administrator’s Guide for Rational ClearCase. If you replicate a component VOB, you must replicate its PVOB. When you use ClearCase UCM and MultiSite, some developer and project manager tasks are different. A project’s integration stream is mastered by one of the replicas in the VOB family, and developers at other replicas must do a remote deliver of their work to the stream. The project manager at the master replica completes the deliver operations. The Developing Software and Managing Software Projects manuals describe this scenario in more detail. 2.4 MultiSite Use Model The following sections describe the different aspects of your MultiSite use model. Type of Administration While you are planning your implementation, you need to decide how much control the individual sites will have over their replicas. Your choices are centralized administration, individual administration, or some combination of the two. ➤ 38 With centralized administration, there is a hub site. For each VOB family, all the replicas in the family are mastered by a replica at the hub site. Administrators at the hub site maintain all replicas and all synchronization patterns and schedules. These administrators have permission to access the VOB replica servers at all sites. Administrator’s Guide: Rational ClearCase MultiSite Advantages of this scheme: ➣ Your company does not have to hire a MultiSite administrator for each site. ➣ It is easier to make sure schedules do not conflict with each other. Disadvantages: ➤ ➣ Some administrative procedures require a replica to be self-mastering. ➣ If ClearCase administration is done at a local level, the MultiSite administrators must have knowledge of all local administrative procedures (for example, backups and server maintenance). ➣ Remote access to all sites is required. With individual administration, each replica is self-mastering and there is an administrator at each site. Administrators are responsible for creating and maintaining replicas, synchronization patterns, and synchronization schedules at their sites. Advantages of this scheme: ➣ No mastership changes are required when an administrator needs to change replica properties. ➣ Administrators can ensure that MultiSite administrative procedures do not conflict with ClearCase administration. Disadvantages: ➣ A MultiSite administrator is needed at each site. ➣ Communication among administrators can be difficult if the company has sites in multiple time zones. You can also have semi-centralized administration. For example, you may have MultiSite administrators at sites with major development efforts and give these administrators control over their MultiSite environment. The responsibility for administering smaller sites is distributed among the MultiSite administrators. 2 - Planning a MultiSite Implementation 39 Mastership Strategy The choices you make for your ClearCase use model and MultiSite administration model determine your mastership strategy. Your plan should state which replicas will master branch types, label types, elements, and other VOB objects. After you create the replicas in the VOB family, you can change mastership of objects. For more information, see Enabling Independent VOB Development: Mastership on page 7 and Changing Mastership on page 126. Replica Permission Strategy When you import a replica-creation packet, you must specify whether the new replica is ownership-preserving or non-ownership-preserving. In most cases, your replicas must be non-ownership-preserving. For information about the requirements for creating ownership-preserving replicas, see Element Ownership and Ownership Preservation on page 4. If you plan to create one or more ownership-preserving VOB replicas, follow these steps: 1. At the exporting site, gather the current VOB ownership and group information and send it along with the packets created by mkreplica –export. a. Get the name of the VOB owner and VOB groups, using the cleartool describe command on the VOB object. For example: cleartool describe vob:/vobs/dev versioned object base "/vobs/dev" created 15-Aug-00.14:19:03 by CC Admin (ccadm.user@minuteman) VOB family feature level: 1 VOB storage host:pathname "minuteman:/vobstg/dev.vbs" VOB storage global pathname "/net/minuteman/vobstg/dev.vbs" database schema version: 53 VOB ownership: owner purpledoc.com/ccadm group purpledoc.com/user b. Translate the symbolic names to numbers. On UNIX, become the VOB owner and issue the id command. For example: su ccadm Password: xxxxxx id uid=1083(ccadm) gid=20(user) 40 Administrator’s Guide: Rational ClearCase MultiSite 2. At each importing site, ensure that the user ID, primary group, and secondary groups match the information from the exporting site, in name and number. If they do not match, you must modify the user and group information to prevent import failures due to permissions problems, as described in Ownership Preservation on page 185. If the names are the same and the numbers are different, you must create non-ownership-preserving replicas. Synchronization Method There are multiple methods you can use to transport MultiSite packets. The method you choose depends on how your sites are connected, how quickly you must transfer packets, and how important security is. Table 4 lists the recommended methods for various situations. Table 4 Choosing a Packet Transfer Method Your situation Recommended methods Source of more information Sites are connected with high-speed lines shipping_server Transferring Packets with Store-and-Forward on page 90 shipping_server reference page One or more sites have firewalls Tape, diskette, CD-ROM, e-mail, ftp, shipping_server Using MultiSite through a Firewall on page 96 syncreplica reference page Must transfer packets quickly E-mail, ftp, shipping_server shipping_server reference page, syncreplica reference page No electronic connection between sites Tape, diskette, CD-ROM syncreplica reference page 2 - Planning a MultiSite Implementation 41 Synchronization Pattern The synchronization pattern for a VOB family defines which replicas exchange update packets and the direction of exchange. Your choice of pattern depends on the following factors: ➤ Bandwidth between sites ➤ Network topology ➤ Latency of changes: how quickly changes made at one replica need to be received at another replica in the family ➤ Failure tolerance The following sections describe unidirectional and bidirectional exchanges and the most common synchronization patterns. Directions of Exchange Synchronization can be unidirectional or bidirectional, as shown in Figure 10. Figure 10 Unidirectional and Bidirectional Updating unidirectional replica1 replica2 replica3 bidirectional replica1 replica2 replica3 In most cases, you will use bidirectional updating. Unidirectional updates are suitable in situations like these: 42 ➤ You use a replica as a backup. ➤ Your company supplies source code to another site (or company) for read-only use. Administrator’s Guide: Rational ClearCase MultiSite ➤ A high-security development project uses the same files as a more open project. In this case, the open project sends updates to the high-security project, but no updates are sent in the other direction. However, unidirectional updates carry some risk. For example, an accidental change of mastership cannot be fixed, and restoring from a replica that does not exchange updates directly with the broken replica involves extra work. Also, you must ensure that no work is done accidentally in a read-only replica; do this by creating triggers or locking the VOB to prevent checkouts and creation of metadata. One-to-One and Ring Synchronization Figure 11 One-to-One Synchronization Pattern Figure 12 Ring Synchronization Pattern The simple one-to-one and ring (or round-robin) patterns are simple patterns that are most suitable for small numbers of replicas. As the number of replicas grows larger, the amount of time increases for a change made at one replica to be received at a replica at the other side of the ring. 2 - Planning a MultiSite Implementation 43 One-to-Many Synchronization 44 Figure 13 Single-Hub Synchronization Pattern Figure 14 Multiple-Hub Synchronization Pattern Administrator’s Guide: Rational ClearCase MultiSite Figure 15 Tree Synchronization Pattern Advantages: ➤ More efficient for the spoke and branch replicas, which send to and receive from only one other replica. Disadvantages: ➤ If the hub or root site goes down, all spoke/branch sites must reconfigure their pattern to keep communication going. ➤ If you change the synchronization pattern so that replicas that did not synchronize directly now exchange packets, the first packets that are generated may be too large for the system. To avoid this problem, you can run chepoch –actual regularly among the spoke or branch replicas. 2 - Planning a MultiSite Implementation 45 Many-to-Many Synchronization Figure 16 Many-to-Many Synchronization Pattern Advantages: ➤ For companies with few sites, this pattern keeps each replica’s epoch table the most accurate for all siblings. ➤ If one site is unavailable, the other sites do not have to change their patterns to continue synchronizing. Disadvantages: ➤ Each administrator must maintain more synchronization jobs and spend more time keeping track of packets. Synchronization Schedule The synchronization schedule for a VOB family defines when replicas in the family send and receive updates. The schedule is affected by many factors, including the rate of development at different sites, the connections among sites, and whether you use MultiSite as a backup strategy. Consider the following issues when planning your synchronization strategy: ➤ Rate of development. If you schedule synchronizations frequently, merging is simpler because fewer changes have taken place. Also, you lose less work if a replica is deleted accidentally and you must restore it from backup. Make sure that synchronizations do not overlap with VOB backups. VOBs must be locked while they are being backed up, and the syncreplica command fails if the VOB is locked. 46 Administrator’s Guide: Rational ClearCase MultiSite ➤ Time zone differences. Be sure to take different time zones into account when you send an update or set up automated updates. Figure 22 on page 89 illustrates synchronization taking place among replicas in three time zones. ➤ Use of administrative VOBs. Because local type objects in a client VOB are linked to global type objects in the administrative VOB, we recommend that you synchronize a client VOB and its administrative VOB at the same time. If you do not, users may have trouble accessing type objects. For example, at the Boston site, the client VOB /vobs/dev is linked to administrative VOB /vobs/admin, and both VOBs are replicated to San Francisco and Bangalore. You export update packets to replicas sanfran_hub@/vobs/dev and sanfran_hub@/vobs/admin at 11:00 P.M. local time and export update packets to replicas bangalore@/vobs/dev and bangalore@/vobs/admin at 5:00 A.M. local time. The administrator at San Francisco imports both packets at the same time, as does the administrator at Bangalore. ➤ Use of ClearCase UCM. We recommend that you synchronize a component VOB and its PVOB at the same time. If you do not, users may have trouble accessing baselines and activities and the versions associated with those objects. Use of MultiSite for Backups You can use MultiSite as part of your VOB backup strategy. For more information, see Chapter 11, Backing Up VOBs with MultiSite. Scrubbing Parameters for VOB Replicas When a ClearCase or MultiSite command makes a change to a replica, an oplog entry is recorded in the replica’s database. (See VOB Operations and the Oplog on page 24 for more information on this mechanism.) Also, when you export an update packet, an export_sync record is created for each target replica. These records are stored in the VOB database and are used by the recoverpacket command to reset a replica’s epoch number matrix. You can scrub oplog entries and export_sync records to reclaim disk space and database records, but you must keep them long enough to ensure that you can recover from replica failures and packet losses. The following sections give guidelines for configuring scrubbing frequency. For more information on VOB scrubbing, see the ClearCase vob_scrubber reference page. 2 - Planning a MultiSite Implementation 47 Oplog Scrubbing Oplog entries must be kept in the database for a significant period. In the near term, they are required when the replica generates update packets to be sent to all other replicas. Beyond that, entries may be required to help other replicas recover from catastrophic failures. If no replica can supply these entries, the replica being restored must be re-created. (See Restoring a Replica from Backup on page 191.) Because of the need to use oplog entries during synchronization, your synchronization strategy determines how often oplogs can be scrubbed. By default, an oplog entry is never scrubbed. Do not change this setting until you establish the synchronization pattern in the VOB family and verify that packets are being exported and imported successfully. When it is safe to delete oplog entries for a replica, follow these steps: 1. Coordinate with administrators at other sites to decide how long each site must keep oplog entries. Each site must keep entries for as long as necessary to allow restorereplica operations to complete successfully. The frequency with which you scrub oplogs depends on the following factors: ➣ The pattern of synchronization among replicas in the VOB family ➣ How often the replicas are synchronized Frequency of synchronization refers both to how often updates are exported and to how often they are imported at other sites. Also, consider setting up a verification scheme so you can ensure that packets are processed successfully at other replicas before any oplog entries are scrubbed. ➣ How often you back up the replicas For example, if a VOB is backed up weekly at all sites and you want to be able to restore to the backup from two weeks ago, each replica must keep three weeks of oplog entries. If replicas synchronize weekly, you must assume that the weekly packet hasn’t been sent to the other replica, and add another week. Finally, for extra security, add another month. The result is a scrubbing time of two months. 2. Change the oplog scrubbing parameter for your replica: a. Copy ccase-home-dir/config/vob/vob_scrubber_params (UNIX) or ccase-home-dir\config\vob\vob_scrubber_params (Windows) to the VOB storage directory of the replica. This creates a parameter file specific to the VOB. 48 Administrator’s Guide: Rational ClearCase MultiSite b. Make this new file writable. c. Edit the oplog line in this file. For example, to keep oplog entries for two months (62 days): oplog –keep 62 CAUTION: If a replica’s oplog entries are scrubbed before they are included in an update packet, you cannot export update packets from the replica. This is a serious error and compromises the integrity of the entire VOB family. export_sync Scrubbing export_sync records are not necessary for normal synchronization operation. They are different from export event records, which also record synchronization exports and are included in output from the lshistory command and the History Browser. export_sync records are date-based records used by the recoverpacket command to reset a replica’s epoch number matrix. If you do not use this packet recovery method (because you use chepoch –actual or lsepoch/chepoch), you can scrub these records aggressively. If you use the recoverpacket command, you must keep export_sync records for the number of days that elapse between VOB backups. (See Recovering from Lost Packets on page 182.) By default, the vob_scrubber_params file has no entry for export_sync records, and these records are scrubbed with the same frequency as oplog entries. If you want to scrub export_sync records at a different frequency than oplog entries, you can set the export_sync parameter in the vob_scrubber_params file. For more information, see the vob_scrubber reference page. 2.5 Responsibilities of MultiSite Administrators A MultiSite administrator must do the following: ➤ Help determine and implement the ClearCase and MultiSite use models When a new project is set up, the administrator works with project managers to determine which replicas master various objects. The administrator also changes mastership when necessary, schedules merges, copies triggers from replica to replica, and monitors label creation. 2 - Planning a MultiSite Implementation 49 ➤ Monitor MultiSite synchronization and replica creation Administrators must check the storage bays to make sure that packets are not accumulating. On UNIX, include the administrator’s e-mail address in the ADMINISTRATOR entry in the shipping.conf file. On Windows, include the administrator’s e-mail address in the MultiSite Control Panel. It is important to prevent two or more replicas of the same VOB from being mounted on the same host—one host can belong to only one region and each region can contain only one replica. Accordingly, do not assign public VOB-tags in the same ClearCase registry region to multiple replicas of the same VOB. See VOB Objects and Replica Objects on page 23 for information about how VOBs and VOB replicas are listed in the ClearCase storage registry and Chapter 12, Using MultiSite for Interoperability for information about using multiple replicas at one site. ➤ Monitor ClearCase and system log files Error and status messages are written to the shipping_server_log file and (on Windows) the Event Viewer. For more information about error logs, see Troubleshooting Tips on page 164. ➤ Install new versions of ClearCase and MultiSite and new patches Patches and information about new versions are available on the Rational Software Web site. Install the Mandatory and Recommended patches for your architecture. Compatibility issues for versions of ClearCase and MultiSite are described in the Release Notes for Rational ClearCase and ClearCase MultiSite. ➤ Coordinate issues with all other MultiSite administrators responsible for replicas in the VOB family After initial setup and synchronization of replicas, administrators also must coordinate recovery efforts, which may involve exchanges of update packets, and changes of mastership, which require the administrator at the master replica to transfer mastership to the replica that needs to master the objects. We recommend that you create a representation of your MultiSite deployment. For example, Figure 17 shows information and the synchronization pattern for a VOB family. 50 Administrator’s Guide: Rational ClearCase MultiSite VOB Family Information Figure 17 Replica name Replica host sanfran_hub goldengate boston_hub minuteman Administrator Email, phone number John Cole jcole, x1462 Susan Goechs susan, x3742 San Francisco GMT-8 Boston GMT-5 Location Time zone offset ➤ tokyo shinjuku sydney taronga bangalore ramohalli buenosaires mardelplata Masako Ito masako, x7761 Bruce Fife bfife, x5080 Sonia Kumar kumar, x2347 Fangio Erizo fangio, x4300 Tokyo GMT+9 Sydney GMT+10 Bangalore GMT+5:30 Buenos Aires GMT-3 Ensure that VOB replicas receive any necessary special handling Restoring a VOB replica’s storage directory from backup is a significant event in the life of a VOB family. Failure to follow the procedure described in the section Restoring a Replica from Backup on page 191 leads to irreparable inconsistencies among the VOB’s replicas. There are no special requirements for backing up a VOB replica’s storage directory. Use the instructions in the Administrator’s Guide for Rational ClearCase for backing up a VOB. Other ClearCase administrative procedures require special steps for replicated VOBs. The procedures in the Administrator’s Guide for Rational ClearCase describe these steps. 2 - Planning a MultiSite Implementation 51 52 Administrator’s Guide: Rational ClearCase MultiSite MultiSite Command Set 3 3 This chapter summarizes MultiSite commands and the ClearCase commands that display MultiSite information. Reference pages for the MultiSite commands are available in Chapter 13, MultiSite Reference Pages, and are also available online: 3.1 ➤ On UNIX, the MultiSite multitool man command displays MultiSite reference pages in either HyperHelp or ASCII format. ➤ On Windows, the MultiSite multitool man command displays reference pages in Windows Help. ➤ On both platforms, the MultiSite Help file includes the MultiSite reference pages in this manual. Location of MultiSite Programs The MultiSite installation places programs and configuration files in the ClearCase installation area on a host. (ccase-home-dir refers to both the ClearCase and MultiSite installation directory). On UNIX, MultiSite programs are located in the ccase-home-dir/bin, ccase-home-dir/etc, and ccase-home-dir/config/scheduler/tasks directories. On Windows, MultiSite programs are located in ccase-home-dir\bin and ccase-home-dir\config\scheduler\tasks. 3 - MultiSite Command Set 53 3.2 multitool Use The multitool program is very similar to the ClearCase cleartool program: ➤ It has a set of subcommands that perform product functions, such as replica creation, synchronization, and management; mastership of objects stored in VOB databases; and failure recovery. Some multitool subcommands are also available in cleartool. ➤ Command options can always be abbreviated to three characters and sometimes fewer, as indicated in the reference pages. ➤ You can use multitool in single-command mode. For example: multitool rename replica:original boston_hub Also in interactive mode: multitool multitool> rename replica:original boston_hub multitool> quit ➤ It has online help facilities. The help command displays syntax summaries, and the man command displays reference pages: multitool help chreplica Usage: chreplica [-c comment | -cfile pname | -cq | -cqe | -nc] [-host hostname] [-preserve | -npreserve] [-isconnected | -nconnected] replica-selector multitool man chreplica ...on Windows, Windows Help displays the reference page chreplica ========== Changes the properties of a replica APPLICABILITY ... 54 Administrator’s Guide: Rational ClearCase MultiSite multitool Subcommands The following sections describe the different kinds of multitool subcommands. The tables in each section show whether the command has a cleartool equivalent and whether a view context is required when you invoke the command. Commands Copied from ClearCase These commands were copied from cleartool and are documented only in the Command Reference, except for apropos, which is also documented in this manual. Table 5 multitool Subcommands Copied from ClearCase Command cleartool equivalent View context required? Description apropos (UNIX) Yes No Displays multitool command information cd Yes No Changes current working directory describe Yes Yes (file-system objects) Describes a replica’s VOB database object help Yes No Displays multitool command syntax man Yes No Displays a MultiSite reference page pwd Yes No Prints working directory quit Yes No Ends interactive multitool session rename Yes No Renames a replica shell Yes No Creates subprocess to run shell or program Replica Creation, Synchronization, and Management multitool includes commands that set up new replicas of VOBs, change their characteristics, and change their contents by importing update packets. 3 - MultiSite Command Set 55 Table 6 Replica Creation, Synchronization, and Management Commands Command cleartool equivalent View context required? Description chreplica No No Changes the properties of a replica lspacket No No Lists one or more packet files created by mkreplica or syncreplica lsreplica Yes No Lists one or more of a VOB’s replicas mkreplica No No Creates a new VOB replica rename Yes No Renames a replica (command documented in the Command Reference) rmreplica No No Removes a replica syncreplica No No Synchronizes the current replica with one or more other replicas in its VOB family Object Mastership To prevent conflicting changes from occurring at different replicas of a VOB, certain VOB-database objects are assigned a master replica (master). The initial master of an object is the replica where the object is created. For more information on mastership, see Enabling Independent VOB Development: Mastership on page 7. Table 7 56 Object Mastership Commands Command cleartool equivalent View context required? chmaster Yes Yes (file-system objects) Transfers mastership of a ClearCase object lsmaster Yes Yes Lists objects mastered by a replica reqmaster Yes Yes Requests mastership or set access controls for mastership requests Description Administrator’s Guide: Rational ClearCase MultiSite Failure Recovery Each replica of a VOB uses an epoch number matrix to track its own state and the state of all other replicas. (Because replicas are always changing, a replica knows what changes have been made to itself; but it can have only an estimate of the states of other replicas.) Each time a replica sends an update packet, it updates its own epoch number matrix, under the assumption that the packet will be delivered to its destinations and applied to the appropriate replicas. For more information, see VOB Operations and the Oplog on page 24. multitool includes the following failure-recovery commands, for use when this assumption of successful delivery does not hold true: Table 8 Failure-Recovery Commands Command cleartool equivalent View context required? Description chepoch No No Changes a replica’s epoch number matrix lsepoch No No Lists a replica’s epoch number matrix recoverpacket No No Resets epoch number matrix so lost packets are resent (required when a packet is lost or unusable) restorereplica No No Restores VOB replica from backup. This command places a replica in a special state, in which it sends epoch number matrix corrections to other replicas. The replica cannot be used for normal development work until it receives special updates that inform it of the current states of other replicas. 3 - MultiSite Command Set 57 3.3 View Contexts and VOB Mounts The principal MultiSite commands do not require a view context or mounting of the VOB replicas being processed. This facilitates use by administrators and automation of MultiSite operations through the schedule command. There are some advantages to running MultiSite commands in a view, with the VOB mounted: 3.4 ➤ Simpler command syntax. If your current working directory is within a VOB, many commands process that VOB, eliminating the need to use the @vob-selector suffix in command arguments. ➤ Better diagnostics. If a syncreplica –import command fails when running in a view, it produces diagnostics that include pathnames, which makes troubleshooting easier. Specifying VOBs and Replicas in Commands ClearCase commands use the vob: prefix to operate on the current VOB replica. ClearCase and MultiSite commands use the @vob-selector suffix to specify the replica that is mounted at a particular VOB-tag. This suffix indicates which replica’s database is to be used by the command. The multitool mkreplica command uses the –vreplica option to specify a particular replica within a VOB family. 58 Administrator’s Guide: Rational ClearCase MultiSite 3.5 Additional MultiSite Commands The MultiSite commands that are not built in to multitool are listed in Table 9. Table 9 Additional MultiSite Commands Command Location under ccase-home-dir Description epoch_watchdog config/scheduler/tasks (UNIX) config\scheduler\tasks (Windows) Checks whether a replica’s epoch numbers have rolled back when the replica is not in restoration mode; for use in schedule commands. mkorder etc (UNIX) bin (Windows) Creates shipping order for use by store-and-forward. notify bin Mail program for store-and-forward. shipping_server etc (UNIX) bin (Windows) Store-and-forward packet transport server. sync_export_list config/scheduler/tasks (UNIX) config\scheduler\tasks (Windows) Replica-update script using store-and-forward; for use in schedule commands. sync_receive config/scheduler/tasks (UNIX) config\scheduler\tasks (Windows) Replica-update script using store-and-forward; for use in schedule commands and as the receipt handler. 3 - MultiSite Command Set 59 3.6 ClearCase Commands Related to MultiSite The ClearCase commands in Table 10 manage or display MultiSite information. Table 10 ClearCase Commands Related to MultiSite Command Description checkout –unreserved –nmaster Performs a nonmastered checkout, which is an unreserved checkout on a branch not mastered by your current replica. lscheckout –areplicas Lists checked-out versions across all replicas of a VOB (Default: lists your current replica’s checkouts). mkattype –shared mkhltype –shared mklbtype –shared Creates a shared type object. mkelem –master Assigns mastership of the main branch of the element to the replica in which you create the element. Also, if your config spec contains mkbranch rules and you do not specify the –nco option with mkelem, mkelem assigns mastership of these branches to the replica in which you create the element. vob_scrubber Scrubs oplog entries and export_sync records. In general, all ClearCase commands obey MultiSite mastership restrictions in a replicated VOB. In addition, the following commands work differently in replicated VOBs: describe Lists the master replica of an object. For replicas, branch types, and branches, lists the mastership request setting. describe vob:pname-in-vob Lists the replica name and the VOB family feature level. 60 Administrator’s Guide: Rational ClearCase MultiSite ln mkelem rmname To change a directory, you must work in the master replica of the branch on which the directory is checked out. Changes to directories include ➤ ➤ ➤ Creating a VOB hard link or VOB symbolic link (ln) Creating a new element (mkelem) Removing a reference to an element or VOB symbolic link (rmname) mk**type –replace If a type object is shared, you cannot change its instance restrictions. For example, you cannot replace a one-per-element branch type with a one-per-branch branch type. mkeltype –replace You cannot change the definition of an element type in a replicated VOB. rmtype eltype:type-name You cannot delete an element type in a replicated VOB. 3 - MultiSite Command Set 61 62 Administrator’s Guide: Rational ClearCase MultiSite Using MultiSite Creating Replicas 4 4 This chapter describes how to plan and create VOB replicas. Before creating a replica, you must make decisions about branching, mastership, ownership preservation, and method of packet delivery. Be sure to read ClearCase Use Model on page 35 and MultiSite Use Model on page 38. 4.1 Overview of Replica Creation You use this three-phase procedure to create new VOB replicas: 1. Export phase—At one site, enter a mkreplica –export command, which creates a new replica object and a replica-creation packet. 2. Transport phase—Send the packet to one or more other sites. 3. Import phase—At the other sites, each administrator enters a mkreplica –import command, which creates a new VOB replica. The procedure is similar for different methods of packet delivery and for different platforms. The example in this chapter assumes a high-speed connection between sites, and all replicas are located on UNIX machines. The procedure is the same if all replicas are located on Windows machines or if one replica is on a Windows machine; only the VOB-tags and pathnames are different. If some replicas in your VOB family will be located on UNIX machines and others will be on Windows machines, be sure to read Replicating a VOB Between UNIX and Windows on page 77. 4 - Creating Replicas 65 4.2 Timing of Replica Creation During the export phase of replica creation, the replica creation command locks the VOB and dumps the VOB database. The VOB is locked for the entire length of time the command runs. While the VOB is locked, read-only operations can occur in the VOB, but write operations cannot. (For example, these operations fail: checkins and checkouts, chepoch –actual commands, label creation, builds, imports of update packets, VOB snapshots, and scheduled backups.) Therefore, you need to schedule the export phase of replica creation during nonbusiness hours for your site. You must also cancel any scheduled exports, imports, VOB snapshots, and backups for the duration of the export phase. 4.3 Notes on Different Transport Methods If your sites have a high-speed connection, you can take advantage of the MultiSite store-and-forward facility when you create a new replica. If your current site does not have IP connectivity to the site of the new replica, you can use a file-based packet transfer method (for example, ftp or email). Store-and-Forward Method The following sections describe issues you must consider when you use the store-and-forward method. Communication Between Replica Hosts The hosts must be able to communicate with each other. If your network uses host names, the sending host must be able to resolve the receiving host’s name to an IP address. To accomplish this, you may have to update the hosts file, hosts NIS map, or Domain Name Service. Verify TCP/IP access by using rcp on each host to access the other hosts. If hosts in your network are known only by their IP addresses, you can use the IP addresses instead of host names, and no resolution is necessary. NOTE: 66 Administrator’s Guide: Rational ClearCase MultiSite Limiting the Size of a Packet The mkreplica command fails if it tries to create a packet larger than the size supported by your system. To prevent this problem and improve reliability, use the –maxsize option to divide the replica-creation packet into multiple packets: multitool mkreplica –export –maxsize 1g ... For information about default packet size limits, see the mkreplica reference page. Transport Options When you enter the mkreplica –export command, you can use either the –fship option to send the packet immediately, or the –ship option to store the packet in the outgoing shipping bay. With –ship, you must invoke the shipping_server to send the packet. The outgoing packet is stored in the outgoing subdirectory of a storage bay. By default, mkreplica uses the default storage bay (ccase-home-dir/shipping/ms_ship on UNIX and ccase-home-dir\var\shipping\ms_ship on Windows). The incoming and outgoing subdirectories of storage bays contain packets waiting for transport or processing. All shipping operations look for packets in these subdirectories. At the receiving site, the incoming packet is stored in the incoming subdirectory of a storage bay. Notes on Using Tape or a File-Based Transfer Method When you use the –tape option (UNIX) or a file-based method for transport, you may need to use the –maxsize option to prevent the tape from filling up or to make sure the file is a manageable size. In this example, the administrator writes the replica-creation packet to tape, using the –maxsize option. The mkreplica command prompts for additional tapes if necessary. 4 - Creating Replicas 67 MINUTEMAN% multitool mkreplica –export –work /usr/tmp/wk –tape /dev/tape \ –maxsize 75m goldengate:sanfran_hub@/vobs/dev Enabling replication in VOB. Comments for "sanfran_hub": First time replication for dev VOB; Creating new replica, sanfran_hub, on host goldengate . Please insert a tape to hold packet number 1. When ready, enter ‘proceed’ (proceed/abort) [proceed] Generating packet number 1... Dumping database... . . . Dumper done. 4.4 Replica-Creation Scenario The replica-creation example in this section uses a fictional company whose software development takes place in Boston, Massachusetts and in a new development office in San Francisco, California. Work is about to begin on a new release. Planning the Rules of the Road The organization uses a common ClearCase software development strategy: ➤ Individual subprojects, and often individual developers, use separate subbranches. The auto-make-branch facility is used in all config specs, to place changes on the appropriate branches. For example: element element element element element ➤ 68 * * * * * CHECKEDOUT .../sanfran_main/LATEST SANFRAN_BASE -mkbranch sanfran_main V1.0 -mkbranch sanfran_main /main/0 -mkbranch sanfran_main The v2.0_integration branch is reserved for integration of the work done at the various sites. To prepare for an internal baseline or an external release, the project manager merges selected development subbranches into the v2.0_integration branch. Administrator’s Guide: Rational ClearCase MultiSite ➤ When necessary, developers merge changes from the v2.0_integration branch to their subbranches, to bring themselves up to date with changes occurring on the integration branch. With Rational ClearCase MultiSite, the organization can continue to use this strategy. The Boston replica masters the v2.0_integration branch. The San Francisco replica masters a branch named sanfran_main, as well as any additional subbranches of sanfran_main that may be needed to organize the work there. The project manager at the Boston site merges changes from the sanfran_main and boston_main branches into the v2.0_integration branch, so that the release engineers can build the product using the latest changes. Relevant characteristics of the two replicas: Boston replica Host name: Replica name: VOB storage directory: VOB-tag: Config spec for development: Config spec for integration: San Francisco replica Host name: Replica name: VOB storage directory: VOB-tag: Config spec for development: 4 - Creating Replicas minuteman (UNIX) boston_hub /vobstg/dev.vbs /vobs/dev element element element element element element element element element element * * * * * * * * * * CHECKEDOUT .../boston_main/LATEST BOSTON_BASE -mkbranch boston_main V1.0 -mkbranch boston_main /main/0 -mkbranch boston_main CHECKEDOUT .../v2.0_integration/LATEST BOSTON_BASE -mkbranch v2.0_integration V1.0 -mkbranch v2.0_integration /main/0 -mkbranch v2.0_integration goldengate (UNIX) sanfran_hub /vobstg/dev.vbs /vobs/dev element element element element element * * * * * CHECKEDOUT .../sanfran_main/LATEST SANFRAN_BASE -mkbranch sanfran_main V1.0 -mkbranch sanfran_main /main/0 -mkbranch sanfran_main 69 The company has not yet merged its user/group databases, so the two replicas cannot be ownership-preserving. There is IP connectivity between the two offices, so the Boston administrator can use the MultiSite store-and-forward facility to create the new replica. Prerequisites Before you create a new replica, you must perform these steps at the original site: 1. Make sure MultiSite licenses are installed. After you enter the mkreplica –export command, developers at the original site cannot access the VOB without a MultiSite license (in addition to a ClearCase license). clearlicense –product MultiSite Licensing information for MultiSite. License server on host "cclicense". Running since Thursday 07/01/00 12:27:28. LICENSES: Max-Users 300 ... Expires none Password [status] 34ms5678.901234c5.67 [Valid] 2. Apply a version label, from which development work at the new replica will branch. In the standard ClearCase manner, a consistent set of source versions (a baseline) is identified by a version label. The VOB administrator creates label type SANFRAN_BASE and attaches it to all the /main/LATEST versions in the original VOB. The changes at sanfran_hub are made on sanfran_main branches; all these branches are created at SANFRAN_BASE versions. 3. Rename the original replica appropriately. Even though the original VOB has not yet been replicated, its VOB database still has a VOB replica object, named original: MINUTEMAN% cleartool lsreplica –invob /vobs/dev For VOB replica "/vobs/dev": 15-Aug.14:19 susan replica "original" 70 Administrator’s Guide: Rational ClearCase MultiSite The administrator renames the VOB replica object to boston_hub: MINUTEMAN% multitool rename replica:original boston_hub Renamed replica from "original" to "boston_hub". MINUTEMAN% cleartool lsreplica –invob /vobs/dev For VOB replica "/vobs/dev": 15-Aug.14:19 susan replica "boston_hub" 4. Make sure the VOB is not locked. Step #6 locks the VOB; an error occurs if the VOB is already locked. MINUTEMAN% cleartool lslock vob:/vobs/dev MINUTEMAN% (null output indicates VOB is not locked) 5. Determine the size of the VOB database and source pools. The directory you specify with the –workdir option must be on a partition that has enough free space to hold the VOB database and the VOB source pools. You must have write permission on its parent directory, and the directory you specify must not exist. To determine the size of the VOB database and source pools, use cleartool space: cleartool space /vobs/dev Use(Mb) ... 1429.0 ... 189.5 ... %Use 17% 2% Directory VOB database /vobstg/dev.vbs/db source pool /vobstg/dev.vbs/s/sdft In this example, the work directory must have at least 1.62 GB of free space. Export Phase These steps take place at the original site. 6. Enter the export form of the replica-creation command. See the mkreplica reference page for information about restrictions on the command. In this example, the administrator uses the –fship option to send the packet immediately. 4 - Creating Replicas 71 MINUTEMAN% multitool mkreplica –export –work /tmp/ms_wkdir \ –fship goldengate:sanfran_hub@/vobs/dev Enabling replication in VOB. Comments for "sanfran_hub": First time replication for dev VOB Creating new replica, sanfran_hub, on host goldengate . Generating replica creation packet /usr/atria/shipping/ms_ship/outgoing/repl_boston_hub_16-Aug-00.09.49.36_14 075_1 - shipping order file is /var/adm/atria/shipping/ms_ship/outgoing/sh_o_repl_boston_hub_16-Aug-00.09 .49.36_14075_1 Dumping database... ... Dumper done. Attempting to forward/deliver generated packets... -- Forwarded/delivered packet /usr/atria/shipping/ms_ship/outgoing/repl_boston_hub_16-Aug-00.09.49.36_14 075_1 7. Back up the original VOB. This backup records the fact that the VOB is replicated. If you have to restore a VOB replica from a backup copy that was made before the VOB was replicated, the MultiSite replica restoration procedure fails. (Although the restorereplica command may succeed, you will not be able to import update packets from other replicas because the original VOB is marked as unreplicated.) 8. (optional) Verify the replica-related changes. These commands show the work you’ve done so far. The mkreplica command creates a new replica object in the VOB database. This object is similar to the VOB object that represents the entire VOB in the database, and its properties are listed by the lsreplica command. MINUTEMAN% multitool lsreplica –invob /vobs/dev For VOB replica "/vobs/dev": 15-Aug.14:19 susan replica "boston_hub" 16-Aug.09:49 susan replica "sanfran_hub" MultiSite commands process replica objects similarly to the way that ClearCase commands process type objects. The rename command renames a replica object, as described in Step #3. The cleartool lshistory lists events associated with replica objects. 72 Administrator’s Guide: Rational ClearCase MultiSite MINUTEMAN% cleartool lshistory replica:boston_hub@/vobs/dev 16-Aug.09:45 susan rename replica "boston_hub" "Changed name of replica from "boston" to "boston_hub"." 15-Aug.14:24 susan rename replica "boston_hub" "Changed name of replica from "original" to "boston"." 15-Aug.14:19 susan make attribute "FeatureLevel" on replica "boston_hub" "Added attribute "FeatureLevel" with value 2." 15-Aug.14:19 susan create replica "boston_hub" CAUTION: Do not modify any properties of the new replica at this point. If you must change any properties, you must first import the replica-creation packet at the new site, export an update packet from the new replica, and import the packet at the original site. Transport Phase 9. Send the replica-creation packet to the new site. This process differs depending on the options you used in Step #6: ➣ If you used –fship in Step #6, the packet was sent to the new site immediately. ➣ If you used –ship, you must run shipping_server to send the packet to the new site. For example: MINUTEMAN% /usr/atria/etc/shipping_server –poll ➣ If you used –tape or wrote the packet to a file, you must transport the tape or file to the new site. Import Phase Complete these steps at the new replica’s site. 10. Verify the packet’s arrival by entering the lspacket command on the receiving host. By default, lspacket searches all the MultiSite storage bays for packets. For example, if host goldengate is the receiving host: 4 - Creating Replicas 73 GOLDENGATE% multitool lspacket Packet is: /usr/atria/shipping/ms_ship/incoming/repl_boston_hub_16-Aug-00.09.49.36_14 075_1 Packet type: Replica Creation VOB family identifier is: 12a3456b.78c901d2.e3ab.45:67:89:c0:1d:e2 Comment supplied at packet creation is: Packet intended for the following targets: sanfran_hub The packet sequence number is 1 11. Enter the import form of the replica-creation command. Notes on replica creation: 74 ➣ This replica is not ownership-preserving, so the user who executes this command becomes the owner of the new VOB replica and all elements in it. This user’s primary group is the group for all elements. Typically, administration is easier if this user is not the root user or a member of the ClearCase administrators group. ➣ As described in Step #5, the work directory must have at least 1.62 GB of free space. ➣ You can bypass the prompt step during replica creation by specifying the –vreplica option with the new replica’s name. This example does not specify that option. ➣ You must specify the pathname of the incoming packet as listed by the lspacket command. Administrator’s Guide: Rational ClearCase MultiSite GOLDENGATE% multitool mkreplica –import –npreserve –work /tmp/wk \ –tag /vobs/dev –public –password 1234xyz –vob /vobstg/dev.vbs \ /usr/atria/shipping/ms_ship/incoming/repl_boston_hub_16-Aug-00.09.49.36_14075_1 The packet can only be used to create replica "sanfran_hub" - VOB family is 87f6265b.72d911d4.a5cd.00:01:80:c0:4b:e7 - replica OID is 0eaa6fc3.737d11d4.adbe.00:01:80:c0:4b:e7 Should I create this replica? [no] yes Comments for "sanfran_hub": replica of /vobs/dev from Boston . Processing packet /usr/atria/shipping/ms_ship/incoming/repl_boston_hub_16-Aug-00.09.49.36_14 075_1... Loading database... Dumped schema version is 53 55 events loaded. 77 pass 2 actions performed. Loader done. Registering VOB mount tag "/vobs/dev"... VOB replica successfully created. Host-local path: goldengate:/vobstg/dev.vbs Global path: /net/goldengate/vobstg/dev.vbs VOB ownership: owner purpledoc.com/jcole group purpledoc.com/user 12. Delete the replica-creation packet. (Update packets are deleted automatically.) 13. (Only if new replica is ownership-preserving) Send an update packet to all other replicas in the VOB family. If you create an ownership-preserving replica, inform other replicas in the VOB family of this property. For example, if sanfran_hub were ownership-preserving, you would enter this command: GOLDENGATE% multitool syncreplica –export –c "ownership-preserving" –fship boston_hub ... 14. Make the new replica self-mastering. See Transferring Mastership of a Replica Object on page 128 for the procedure. You must complete this step before you can set the new replica’s feature level or enable requests for branch mastership at the replica. 4 - Creating Replicas 75 15. Set the feature level of the new replica to the feature level of the version of Rational ClearCase running on the replica host. To determine the feature level of the version of ClearCase, enter the cleartool –ver command on the replica host to display the ClearCase version. Then consult the Release Notes for the feature level associated with the version. To set the new replica’s feature level, enter a chflevel command on the replica host: cleartool chflevel –replica feature-level replica-selector For example: cleartool chflevel –replica 2 sanfran_hub@/vobs/dev Replica feature level raised to 2. 16. Send an update packet to all other replicas in the VOB family, to inform them of the new replica’s feature level. For example: GOLDENGATE% multitool syncreplica –export –c "new feature level" –fship boston_hub ... 17. Create a branch type for work in the new replica. The San Francisco developers work on the sanfran_main branch type. GOLDENGATE% cleartool mkbrtype sanfran_main Comments for "sanfran_main": sanfran branch for work on dev project . Created branch type "sanfran_main". Subbranches named sanfran_main are created as necessary. The following config spec automates the creation of the sanfran_main branches: element element element element element * * * * * CHECKEDOUT .../sanfran_main/LATEST SANFRAN_BASE -mkbranch sanfran_main V1.0 -mkbranch sanfran_main /main/0 -mkbranch sanfran_main This config spec is defined in terms of a branch type (sanfran_main) that is mastered by replica sanfran_hub, and label type (SANFRAN_BASE and V1.0) that are mastered by replica boston_hub. The San Francisco developers cannot make any changes in the SANFRAN_BASE labels, but there is no reason for them to do so. 76 Administrator’s Guide: Rational ClearCase MultiSite 18. Verify the mastership of the new branch type. GOLDENGATE% cleartool describe brtype:sanfran_main@/vobs/dev branch type "sanfran_main" created 16-Aug-00.18:12:23 by John Cole (jcole.user@goldengate) "sanfran branch for work on dev project" master replica: sanfran_hub@/vobs/dev ... 19. (For IP-connected replicas) At each site, mark any sibling replicas that have IP connectivity to the current replica. For more information, see Setting the Connectivity Property on page 116. At the boston_hub replica: multitool chreplica –isconnected sanfran_hub@/vobs/dev Updated replica information for "sanfran_hub". At the sanfran_hub replica: multitool chreplica –isconnected boston_hub@/vobs/dev Updated replica information for "boston_hub". 20. Begin development. Developers in San Francisco can access the new replica in the same way they would access an unreplicated VOB. 4.5 Replicating a VOB Between UNIX and Windows This section describes issues involved in setting up UNIX and Windows replicas at different sites. If you plan to use MultiSite at a single location for interoperability between UNIX and Windows, see Chapter 12, Using MultiSite for Interoperability. If your sites do not have an IP connection, you must use electronic mail, CD-ROMs, tapes, or diskettes to transfer packets. You may have to solve compatibility problems if you choose to use tapes or diskettes. With electronic mail, you can use compatible encoding and compression methods. However, differences between UNIX and Windows VOBs are handled automatically during packet import. The most important problems you must prevent are file names that differ only in how they are capitalized, and differences in use of line terminators. If case-sensitive file names are used at one 4 - Creating Replicas 77 replica and case-insensitive file names are used at another replica, errors can occur during synchronization. Differences in use of line terminators between UNIX and Windows editors cause unexpected behavior during file comparisons and merges. Even if the contents of the files are identical, different line terminators indicate differences in the files and require a merge. The Administrator’s Guide for Rational ClearCase describes these problems and their solutions in detail. Be sure to read it before setting up UNIX and Windows replicas. 78 Administrator’s Guide: Rational ClearCase MultiSite ClearCase Feature Levels 5 5 This chapter describes ClearCase feature levels and how to raise the feature level of a replica and a VOB family. 5.1 Overview of Feature Levels Feature levels allow different replica hosts in a VOB family to run different versions of Rational ClearCase. New versions of ClearCase may introduce features that are incompatible with old versions, but administrators may not be able to upgrade all replica hosts at the same time. Feature level control enables you to upgrade replica hosts at different times and to prevent developers from using new ClearCase features that are not meaningful to replicas on hosts running earlier versions. Each version of ClearCase has a feature level. Each VOB family has a single feature level called the family feature level. Each replica in the family has a feature level called the replica feature level. Thus, each VOB family has one family feature level and possibly several replica feature levels. The family feature level determines which ClearCase features can be used by all of the replicas in the family. The following constraints are enforced: ➤ The replica feature level is less than or equal to the feature level of the version of ClearCase installed on the replica’s server host. Different replicas on the same server host can have different feature levels. ➤ The family feature level is less than or equal to the lowest replica feature level found among replicas in the VOB family. Figure 18 shows an example. 5 - ClearCase Feature Levels 79 Figure 18 VOB Family Feature Levels VOB Family Feature Level <=1 VOB Family Feature Level <=2 sanfran_hub FL=2 boston_hub FL=1 tokyo FL=2 buenosaires FL=1 bangalore FL=2 sydney FL=2 The general procedure for raising a family's feature level is as follows: 1. Install the new version of ClearCase on the server hosts of replicas in the VOB family. 2. Raise the feature level of each replica in the VOB family. See Raising the Replica Feature Level. 3. Raise the feature level of the VOB family. See Raising the VOB Family Feature Level on page 82. You can complete these steps incrementally and over a period of days or weeks, if necessary. Variations are possible; for example, if a VOB family has replicas R1 and R2 on servers S1 and S2, respectively, you can install a new version of ClearCase on S1 and raise R1's replica feature level before installing the new version on S2. However, you can complete Step #3 only after you have raised all replicas in the family to the new feature level. For information about the feature level associated with the current version of ClearCase, and the list of features that are disabled until the VOB family feature level is raised, see the Release Notes for Rational ClearCase and ClearCase MultiSite. 5.2 Raising the Replica Feature Level There are two important rules related to raising a replica's feature level: 1. If the current family feature level is less than or equal to 1, the first replica in a VOB family whose feature level is raised must be the replica that masters the VOB object. 2. The replica must be self-mastering. 80 Administrator’s Guide: Rational ClearCase MultiSite To raise the replica feature level: 1. After installing the new version of ClearCase on a server host, determine which replica masters the VOB object: cleartool describe vob:vob-tag If the replica whose feature level you want to raise first does not master the VOB object, transfer mastership, and then export an update packet to the replica whose feature level you want to raise: multitool chmaster replica-name vob:vob-tag multitool syncreplica –export –fship replica-name@vob-tag At the receiving replica, import the packet: multitool syncreplica –import –receive 2. Determine whether the replica is self-mastering: cleartool describe replica:replica-name@vob-tag 3. If the replica is not self-mastering, convert it to a self-mastering replica. See Transferring Mastership of a Replica Object on page 128. 4. Raise the feature level of the replica. Enter this command on the replica host: cleartool chflevel –replica feature-level replica:replica-name@vob-tag 5. Export update packets to all other replicas in the VOB family. 6. (optional) Change mastership of the replica back to the original master replica. 5 - ClearCase Feature Levels 81 5.3 Raising the VOB Family Feature Level There are two variants of the procedure for raising the family feature level: ➤ Raising the feature level of a VOB family in which all replicas send update packets (bidirectional synchronization). See VOB Families with Bidirectional Synchronization on page 82. ➤ Raising the feature level of a VOB family in which one or more replicas receive update packets, but do not send them (unidirectional synchronization). See VOB Families with Unidirectional Synchronization on page 82. VOB Families with Bidirectional Synchronization After raising the feature level of all replicas in the VOB family: 1. Raise the family feature level. Enter this command at the replica that masters the VOB object: cleartool chflevel –family feature-level vob:vob-tag 2. Export update packets to all replicas in the family. VOB Families with Unidirectional Synchronization In some VOB families, one or more replicas may be one-way replicas. These replicas import packets, but they do not export packets to any other replicas in the family, and therefore cannot communicate changes in feature level. Because other replicas in the family do not know the current feature level of the one-way replicas, the chflevel –family command fails. For example, consider the case of two replicas, R1 and R2, that constitute a VOB family. R1 sends update packets to R2, but R2 does not send update packets to R1. R1 82 R2 Administrator’s Guide: Rational ClearCase MultiSite R1 is at replica feature level 2, and R2 is at replica feature level 1. Therefore, the family feature level is 1 and cannot be raised. Now suppose R2’s replica feature level is raised to 2. R2 cannot communicate the change in feature level to R1 because it does not export update packets. Because both replicas are now at feature level 2, the VOB family feature level can be raised to 2. However, if the R1 administrator issues the command chflevel -family 2 vob-selector, the change fails because R1 doesn’t know that the replica feature level at R2 has been raised. In this case, the R2 administrator must inform the R1 administrator of the change in R2’s replica feature level. The R1 administrator then uses a special form of the chflevel command to raise the VOB family feature level. The general procedure is as follows: 1. The administrator of a one-way replica notifies other replica administrators in the VOB family of a change in replica feature level at the one-way replica. 2. At the replica that masters the VOB object, the administrator enters the following command: cleartool chflevel –force –override –family feature-level vob:vob-tag CAUTION: This form of the chflevel command bypasses the constraint that the family feature level is no higher than the lowest known feature level of the replicas in the VOB family. Use it only when you are certain that all replicas in the VOB family are at the same feature level. If you use this command inappropriately, synchronization will fail. 3. At the replica that masters the VOB object, export update packets to all replicas in the family. 5.4 Displaying Feature Levels To display the feature level of a replica: ➤ Use the command cleartool describe replica:replica-name@vob-tag. For example: cleartool describe replica:tokyo@\dev replica "tokyo" created 20-Aug-00.13:35:37 by John Cole (jcole@goldengate) replica type: unfiltered master replica: sanfran_hub@\dev ... feature level: 2 ... 5 - ClearCase Feature Levels 83 ➤ On Windows, open the Properties Browser for the replica. To display the feature level of a VOB family, use the command cleartool describe vob:vob-tag. For example: cleartool describe vob:/vobs/dev versioned object base "/vobs/dev" created 15-Aug-00.14:19:03 by Susan Goechs (susan.user@minuteman) master replica: boston_hub@/vobs/dev replica name: boston_hub VOB family feature level: 2 ... Before you set the feature level for a newly created replica, its value is recorded as unknown. For example, if you use the describe command to show the properties of a new replica, the output looks like this: NOTE: cleartool describe replica:sanfran_hub@/vobs/dev ... feature level: unknown 5.5 Feature Levels Error Message The following error message is printed when a user attempts to use a feature that is not meaningful to sibling replicas: The feature level of the VOB family is not high enough to permit this operation. 84 Administrator’s Guide: Rational ClearCase MultiSite Synchronizing Replicas 6 6 This chapter describes the process of synchronization. Synchronization uses the same export-transport-import procedure that is used during replica creation: ➤ Export phase—At one site, a syncreplica (synchronize replica) command is invoked with the –export option. This creates a packet of data. ➤ Transport phase—The packet is sent to one or more other sites. ➤ Import phase—At the other sites, a syncreplica command is invoked with the –import option. This applies the changes in the packet to an existing replica. The syncreplica command is optimized for performance; it creates a packet that contains only the information required to update the target replicas specified on the command line. Figure 19 illustrates the MultiSite replica-synchronization scheme. At Site 1, a syncreplica –export command places version data and records of operations from replica1 into a packet. The packet is sent to Site 2. At Site 2, a syncreplica –import command imports the contents of the packet into replica2. Note that each synchronization is one-way. If two replicas update each other, two synchronizations are required. 6 - Synchronizing Replicas 85 Figure 19 Replica Synchronization Site 1 replica1 Export outgoing packet Transport Site 2 incoming packet 6.1 Import replica2 Synchronization Patterns Figure 19 shows a simple case, involving one point-to-point update. All updates need not be point to point, however, because they are cumulative. Suppose that the following updates take place among three replicas: Update 1: Update 2: Replica 1 sends changes to Replica 2 Replica 2 sends changes to Replica 3 There is no need for Replica 1 to update Replica 3 directly, because the changes from Update 1 are included in Update 2. This feature gives administrators flexibility in devising update strategies and patterns. For efficiency, a single update can be targeted at multiple sites, for example, all other replicas in the VOB family. In general, you can implement any update topology, as dictated by organizational structures, communications/transportation costs, and so on. Figure 20 shows a simple peer-to-peer synchronization update pattern and Figure 21 shows a double-hub hierarchical pattern. See Chapter 2, Planning a MultiSite Implementation, for more information. 86 Administrator’s Guide: Rational ClearCase MultiSite Figure 20 Peer-to-Peer Synchronization Pattern Boston San Francisco Bangalore Figure 21 Hierarchical Synchronization Pattern Boston Tokyo San Francisco Bangalore Sydney Buenos Aires Designing an Update Strategy Site administrators must design a strategy for sending updates among the various replicas. They must specify an update pattern for the VOB family and an update frequency for each replica. MultiSite Use Model on page 38 gives planning guidelines. 6 - Synchronizing Replicas 87 For example, the administrators for the VOB family in Figure 21 make the following decisions: ➤ The hub replicas, which undergo rapid development, synchronize every hour. ➤ Each hub replica synchronizes daily with its spoke replicas. Each spoke replica will send an update packet to the hub replica, and then the hub replica will send update packets back to the spoke replicas. Because these packets may be large and take a long time to import, the synchronization must not take place during working hours or during VOB backups. Synchronization cannot overlap with VOB backup because VOBs must be locked while they are being backed up, and the syncreplica command fails if the VOB is locked. NOTE: ➤ All sites use receipt handlers to import packets as soon as they are received. Figure 22 shows the synchronization timeline for the hub-spoke updates (but not the hub-to-hub updates). This timeline accounts for time zone differences and includes extra time to make sure that each synchronization phase completes before another begins. 88 Administrator’s Guide: Rational ClearCase MultiSite Figure 22 A Synchronization Export Schedule Monday Tuesday GMT 21:00 00:00 03:00 06:00 09:00 12:00 Bangalore GMT +5:30 Buenos Aires GMT -3 Boston GMT -5 San Francisco GMT -8 Tokyo GMT +9 Sydney GMT +10 Key = work hours/ backup hours = Export 6 - Synchronizing Replicas = Import 89 6.2 Assumption of Successful Synchronization The export and import phases of synchronization always occur at different times. A sending replica does not require acknowledgment from a sibling replica that a packet has been received and processed successfully. Instead, the sending replica assumes success. This enables an optimization: subsequent updates from a replica do not include the data sent in previous updates. If a failure does occur (for example, a packet is lost in transit or a CD-ROM is unreadable at the sibling replica’s site), the sending site must adjust its records to enable the lost data to be resent. For more on this topic, see Chapter 10, Troubleshooting MultiSite Operations. 6.3 Transferring Packets with Store-and-Forward The MultiSite store-and-forward facility is a file-transfer service that automates the transport phase. This facility can handle packets of any size, can route files through a series of MultiSite hosts, one hop at a time, and includes support for handling data-communications failures. The major components of the store-and-forward facility are illustrated in Figure 23 and described in the following sections. To use store-and-forward, the sending host must be able to communicate with the receiving hosts. To determine whether the hosts can communicate, use the rcp command on the sending host to copy a file to the receiving host. If it fails, you may have to update the hosts file, hosts NIS map, or Domain Name Service before using store-and-forward. NOTE: 90 Administrator’s Guide: Rational ClearCase MultiSite Figure 23 The Store-and-Forward Facility Export Phase Import Phase Storage bays Replica Replica Replica Shipping order Packet shipping.conf or MultiSite Control Panel Transport Phase Storage bays Replica Replica Shipping order Packet Transport continues: When shipping_server is invoked on this host, it sends packet on next hop. Packet Storage During the Export and Import Phases When a physical packet file is exported from a VOB replica and submitted to the store-and-forward facility, it is accompanied by a shipping order file, which specifies delivery instructions. The packet is stored in one of the storage bay directories on the VOB replica host. Each storage bay directory contains two directories, incoming and outgoing, which hold the incoming and outgoing packets and their corresponding shipping order files. Shipping operations look in the incoming and outgoing directories for packets. A default storage bay, ms_ship, is created on a host when Rational ClearCase MultiSite is installed there. NOTE: On Windows, the amount of available space on the disk partition where the shipping bays are located must be at least twice the size of the largest packet that will be stored in the shipping bays. There may be two copies of the same packet in the bay at one time: one on its way to another destination and another waiting to be applied to the replica on the host. Return bays are similar to storage bays and provide “return-to-sender” storage for packets that could not be delivered successfully. A default return bay, ms_rtn, is created on a host when MultiSite is installed there. This bay has two subdirectories, incoming and outgoing, which hold the incoming and outgoing packets. Shipping operations look in the subdirectories for packets. 6 - Synchronizing Replicas 91 Packet Transport The shipping_server program transfers packet files from a storage bay (or return bay) at one site to the corresponding bay at another site. An explicit command, manual or automated, invokes the shipping_server on the sending host. The shipping_server process contacts the albd_server process on the receiving host, which in turn invokes the shipping_server on the receiving host in receive mode. After a TCP/IP connection has been established between the sending and receiving invocations of shipping_server, the file is transferred. Configuring the Store-and-Forward Facility The settings for the store-and-forward facility are host-specific. You can specify locations of storage and return bays, routing information to support multihop packet delivery, specifications to handle failure-to-deliver situations, receipt handlers, and so on. For more information on specifying settings, see the shipping.conf reference page on UNIX or the MultiSite Control Panel reference page on Windows. Submitting Packets to Store-and-Forward When you generate a replica-creation or update packet, you can specify that the store-and-forward facility must deliver it. Both syncreplica and mkreplica support the following options: 92 ➤ The –fship option places the packet files and shipping order files in one of the host’s storage bays, and runs shipping_server to send the packet files to their destination host or route them to an intermediate host. ➤ The –ship option places the packet files and shipping order files in a storage bay, but does not invoke shipping_server. The packet files are sent the next time the shipping_server polls the appropriate bay. For information about setting up shipping_server to run automatically, see Automated Synchronization on page 104. Administrator’s Guide: Rational ClearCase MultiSite Sending Files That Are Not Packets You can send any file using the store-and-forward facility if you create a shipping order for the file with the mkorder utility. You can send the file immediately or wait for the shipping_server to send it. ➤ To send a file immediately, use the –fship option with mkorder: /usr/atria/etc/mkorder –data /usr/rptgen/brdcst.0702 –fship –copy boston_hub tokyo ➤ To store the file in a shipping bay so that shipping_server will send the file the next time it runs, use the –ship option: /usr/atria/etc/mkorder –data /usr/rptgen/brdcst.0702 –ship –copy boston_hub tokyo NOTE: The shipping order must be located in the same directory as the file. After you invoke the mkorder command, you can delete the original file. If a file with the same name already exists on the receiving host, the file you send is renamed to filename_1. If you transmit another file with the same name, it is renamed to filename_2, and so on. Setting Up an Indirect Shipping Route The shipping order for a packet includes the host name of the packet’s final destination or several such host names. By default, the store-and-forward facility sends the packet directly to its destination host. You can specify that the packet must be sent to an intermediate host by associating it with a routing hop in the shipping.conf file (UNIX) or in the MultiSite Control Panel (Windows). For example: ➤ On a UNIX host, the shipping.conf file includes this line: ROUTE ➤ sydney_fw sanfran_hub boston_hub tokyo On a Windows host, the Routing Information section in the MultiSite Control Panel specifies host sydney_fw in the Next Routing Hop box and hosts sanfran_hub, boston_hub, and tokyo in the Destination Hostnames box. 6 - Synchronizing Replicas 93 Any packet whose final destination is host sanfran_hub, boston_hub, or tokyo is forwarded to host sydney_fw. At this point, the local host has completed its task, and responsibility for delivering the packet now belongs to sydney_fw. Host sydney_fw can transmit the packet to its final destination directly, or send it to yet another intermediate host, depending on the settings in its shipping.conf file or in the MultiSite Control Panel. NOTE: In a multihop transmission, using the –fship option on the original host causes the first hop to occur immediately. Subsequent hops occur when shipping_server is invoked on the intermediate hosts, which may not be immediately after the packets are received. Retries, Expirations, and Returned Data The shipping_server makes one attempt to transmit a packet to another host. If the packet cannot be transmitted (for example, because the receiving host is unavailable), shipping_server generates an error message and log file entry and exits. Administrators can set up a retry scheme to control its frequency: ➤ After successful transmission of a packet, shipping_server deletes the packet and its shipping order. After a failure, the packet and shipping order remain in the storage bay. ➤ shipping_server –poll transmits all packets it finds in one or more storage bays. Thus, any packets that remain after a transmission failure are sent (if possible) by the next invocation of shipping_server –poll. The following job definition performs this operation every hour: Job.Begin Job.Id: 16 Job.Name: "Shipping Server Poll" Job.Description.Begin: Every hour, run the shipping server to send out any outstanding orders. Job.Description.End: Job.Schedule.Daily.Frequency: 1 Job.Schedule.FirstStartTime: 00:00:00 Job.Schedule.StartTimeRestartFrequency: 01:00:00 Job.DeleteWhenCompleted: FALSE Job.Task: 13 Job.Args: -quiet 1 -poll Job.End See the cleartool schedule reference page in the Command Reference and Automated Synchronization on page 104. 94 Administrator’s Guide: Rational ClearCase MultiSite Attempts to transmit an undelivered packet can continue indefinitely, through repeated invocations of shipping_server. However, administrators usually want to fix problems with failed transmissions instead of letting the attempts continue. Accordingly, each shipping order can include an expiration date-time, specified with one of the following: ➤ The command option –pexpire ➤ (UNIX) An EXPIRATION entry in the shipping.conf file ➤ (Windows) A Packet Expiration value in the MultiSite Control Panel at the sending host By default, shipping orders expire 14 days after they are created. When shipping_server encounters a shipping order that has expired, it does not attempt to transmit the corresponding packet to its destination. Instead, it does the following: ➤ It modifies the shipping order to return the packet to the original sending host, where it is placed in a special return bay. ➤ It sends an electronic mail message to one or more addresses on the original sending host. (Another message is sent when the returned packet arrives at the original sending host.) The return trip may involve multiple hops, as described in Setting Up an Indirect Shipping Route on page 93. During such a trip, a packet is placed in the return bay of each intermediate host. Each hop is handled by shipping_server –poll, which processes a host’s return bay in addition to its storage bays. The expiration time for a packet’s return trip is 14 days; a packet that cannot be returned in that interval is deleted. Error Notification in a Mixed Environment If a packet is delivered through a Windows host on which e-mail notification is not enabled, a failure on that Windows host means that no notification message is sent by electronic mail. Instead, a message is written to the event log; this message contains a request that the appropriate users be informed of the failure. For information about enabling e-mail notification, see the MultiSite Control Panel reference page. Differentiating Packets with Storage Classes You can configure the store-and-forward facility to handle updates for different VOBs in different ways. Each packet can be assigned to a storage class, and each storage class can have its own storage bay, return bay, and expiration period. 6 - Synchronizing Replicas 95 On UNIX, a storage class can be assigned several storage bays; in this case, shipping_server uses the size of the packet to select one of the bays. Conversely, several storage classes can share one or more storage bays. NOTE: You can use multiple storage classes to segregate the packets for VOBs belonging to different groups. By adjusting the operating system permissions on the storage bay directories, you can protect the packets from unauthorized use. You can also use a separate storage class when you use the store-and-forward facility to transfer non-MultiSite files between sites. For more information on storage classes, see the shipping.conf and MultiSite Control Panel reference pages. 6.4 Using MultiSite through a Firewall The MultiSite store-and-forward facility cannot operate through a firewall unless you configure MultiSite differently. Passing through a firewall is usually accomplished by granting access via specific ports and IP addresses. Because store-and-forward picks any available port number on each end to make the connection, there is no single port number (or even small range of port numbers) to which special access can be granted. This section describes several ways to use MultiSite through a firewall: ➤ ➤ ➤ ➤ Use an existing electronic mail mechanism as the transport. Use the standard ftp utility to transport packets. Use a custom TCP application. (UNIX) Install the store-and-forward software on a host that can communicate through the firewall. Using Electronic Mail You can use an existing electronic mail mechanism as the transport. On the sending end, compress and encode the update packet; then send the resulting data to a specific mail alias at the receiving site. On the receiving end, redirect the mail alias to a script that decodes and decompresses the incoming information. To ensure that a mail message is not too large to be delivered, you can generate packets no larger than a specific size by using the –maxsize option, the shipping.conf file (UNIX), or the MultiSite Control Panel (Windows). 96 Administrator’s Guide: Rational ClearCase MultiSite Advantages: ➤ ➤ Transport mechanism is well understood and widely available. Little effort is required from the system administrator. Disadvantages: ➤ ➤ ➤ ➤ No control over routing of data. Possibility that messages can be lost without notification. Messages can be intercepted easily. Less efficient than ftp or store-and-forward. Notes: ➤ You can write scripts to automate e-mail transport. The sending script creates the update packets, compresses and encodes them, and divides them into multiple small packets so they are not too big for the e-mail process. The script must mark the multiple packets with the correct sequencing. The script then sends the packets to an address at the target replica. At the target replica, the account that receives the packets redirects or pipes the packets to a process that reassembles, decodes, and uncompresses the packets, and then places them in the replica’s storage bay. MultiSite import commands handle out-of-sequence and missing packet problems, so your scripts do not have to address these issues. ➤ Using ssh and scp (secure shell and secure copy) provides a secure way to move files through firewalls. ➤ For security, you must encrypt the packets. Using FTP The ftp utility can transport packets. On the sending end, the MultiSite administrator or a script creates and compresses the packet, and uses ftp to transfer the file to a location outside the firewall. This location, or dropsite, must be accessible by MultiSite administrators at other sites. Receiving sites poll the dropsite, looking for any new files. When new files arrive, the receiving sites retrieve them via ftp, decompress them, and process them as usual. 6 - Synchronizing Replicas 97 Advantages: ➤ ➤ Transport mechanism is well understood and widely available. More reliable and efficient than electronic mail. Disadvantages: ➤ ➤ ➤ ➤ Use of a dropsite is required. Polling of the dropsite is required. More complicated to implement, due to the interactive nature of the ftp utility. More administration is required because a third system (the dropsite) is used. Using Custom Software A custom TCP application can accept data and send it from one site to a waiting application at another site. Guidelines for simple applications that send data are often described in the network programming documentation provided by the vendor. If the sending and receiving applications use a fixed port number, the administrator can configure the firewall to permit access. Advantages: ➤ ➤ ➤ ➤ Efficient and reliable. No dropsites required. Electronic mail-capable network is not required. Data interception is more difficult. Disadvantages: ➤ ➤ Custom coding is required. Not as flexible as electronic mail or FTP solutions. Installing Store-and-Forward on a UNIX Firewall Host Because of security concerns, we recommend that you use this method only if other methods are unsuitable for your site. This method is not available for Windows firewall hosts. NOTE: An alternative to using mail, ftp, or custom software is to install the store-and-forward software on a “firewall host,” a host that can communicate through the firewall. MultiSite synchronization commands can forward data intended for systems on the other side of the firewall to this host. 98 Administrator’s Guide: Rational ClearCase MultiSite The software on this host then forwards packets through the firewall to the next hop. To specify the range of port numbers to be used on the host, you can use the environment variables CLEARCASE_MIN_PORT and CLEARCASE_MAX_PORT. In Figure 24, the hosts that communicate through the firewall are the firewall hosts; they have the MultiSite store-and-forward software installed on them, but not ClearCase software. The replica server hosts have Rational ClearCase and MultiSite installed on them. Figure 24 Store-and-Forward Configuration FIREWALL replicaB replicaA Site A Site B This section describes issues you must consider before installing MultiSite on a firewall host and gives instructions for installation. 6 - Synchronizing Replicas 99 Firewall Issues Before enabling shipping_server on a firewall host, consider the following issues: ➤ Shipping bays can be overfilled. Using shipping_server on a firewall host enables anyone coming in from the network to fill shipping bays on the local network, on any machine where a shipping_server is available. To avoid full disks and the related problems: ➤ ➣ Ensure that all shipping bays in the local network are on partitions of their own, so that filling the bays does not degrade system performance. ➣ Install shipping_server only on machines that need it: servers with replicated VOBs and machines used by administrators. Packets are susceptible to snooping. In normal update packets, version information is not encoded. Therefore, anyone shipping packets across an unsecured network must encrypt the packets. Also, the format of a update packet is not very complicated; a dedicated programmer could figure out the format and create a packet with operations that damage a VOB. Encrypting the data makes this kind of attack much more difficult. ➤ Other servers can be accessible. Allowing shipping_server access also allows access to all servers created by the albd_server. Because the albd_server assigns port numbers in the allowed range to other servers running locally, programs from the outside network can connect to all of those servers. Therefore, the firewall host that runs the shipping_server must not run other ClearCase servers. If you can specify the ports to which programs can connect and the IP addresses that are allowed to connect, we recommend that you do so. It further limits the possibility that unauthorized machines can breach the firewall. (You specify ports during the firewall configuration process.) Installing shipping_server on a Firewall On UNIX, the ClearCase Product Family installation includes an option to install only the shipping_server software. Follow the instructions in the Installation Guide for the ClearCase Product Family and select only the shipping_server-only option. Do not install ClearCase on the firewall host. On Windows, there is no installation option for installing only the shipping_server software. 100 Administrator’s Guide: Rational ClearCase MultiSite Controlling Ports Used by albd_server and shipping_server The environment variables CLEARCASE_MIN_PORT and CLEARCASE_MAX_PORT specify the range of port numbers that the albd_server and shipping_server can allocate for communication purposes. When the server needs to assign a port number, it starts with the value of CLEARCASE_MIN_PORT and continues through the range until it reaches CLEARCASE_MAX_PORT. If a port in the range cannot be allocated, the server sleeps and then tries the ports again. When shipping_server detects that the port environment variables are set, it tries to use TCP to make the connection with the albd_server on the receiving host. If this connection fails, shipping_server tries UDP. Therefore, if you have TCP connectivity, you do not have to enable UDP or open UDP ports on the firewall host. Running an individual shipping_server does not require more than two ports at a time. When there are multiple requests to be sent, shipping_server forks. Child processes handle individual requests. The shipping_server starts no more than 10 child processes (and starts that many only if there are 10 requests to process simultaneously), so the maximum range is 20 ports. If the range is smaller, it may result in failed attempts, which can be retried later. Guidelines for Setting Port Values The value range for CLEARCASE_MIN_PORT is 1024 through 65534, and the value range for CLEARCASE_MAX_PORT is 1025 through 65535. The value of CLEARCASE_MAX_PORT must be greater than the value of CLEARCASE_MIN_PORT. We recommend that you use the range 49152 through 65535, which is the Dynamic/Private Port Range. If you use a value within the Registered Ports range (1024 through 49151), the shipping.conf parser prints an informational message. NOTE: Specifying Port Values To specify minimum and maximum port values, set the CLEARCASE_MIN_PORT and CLEARCASE_MAX_PORT environment variables in the following places: ➤ The shipping.conf file on the firewall host. For more information, see the shipping.conf reference page. 6 - Synchronizing Replicas 101 ➤ The atria_start script: a. On the firewall host, edit the file ccase-home-dir/etc/atria_start. b. Add the following lines, replacing min-port and max-port with your minimum and maximum port values. These lines must precede the section that starts the albd_server. # # Set values for minimum and maximum port numbers # CLEARCASE_MIN_PORT=min-port CLEARCASE_MAX_PORT=max-port export CLEARCASE_MIN_PORT export CLEARCASE_MAX_PORT 6.5 Manual Synchronization This section describes how to synchronize replicas by entering explicit syncreplica commands. Export Phase 1. Create an update packet. At the sending host, use the syncreplica –export command with the appropriate transport option. If your sites are connected electronically, you can use store-and-forward to send the packet (–fship) or place it in a storage bay (–ship): multitool syncreplica –export –maxsize 1m –fship boston_hub@\dev Generating synchronization packet C:\Program Files\Rational\ClearCase\var \shipping\ms_ship\outgoing\sync_bangalore_30-Oct-00.14.35.49_2468_1 - shipping order file is C:\Program Files\Rational\ClearCase\var\shipping \ms_ship\outgoing\sh_o_sync_bangalore_30-Oct-00.14.35.49_2468_1 Attempting to forward/deliver generated packets... -- Forwarded/delivered packet C:\Program Files\Rational\ClearCase\var \shipping\ms_ship\outgoing\sync_bangalore_30-Oct-00.14.35.49_2468_1 This example uses the –out option to save the packet as an output file and includes the –maxsize option to divide the logical packet into appropriately sized physical packets. The packet files can then be sent by electronic mail or copied onto diskettes. 102 Administrator’s Guide: Rational ClearCase MultiSite multitool syncreplica –export –maxsize 1m –out c:\packets\update1 boston_hub@\dev Generating synchronization packet c:\packets\update1 Transport Phase 2. Send the packets. Use electronic mail, regular mail, or your preferred delivery method. If you used syncreplica –export –ship, invoke shipping_server in either of the following ways: shipping_server –poll shipping_server shipping-order-pathname Import Phase 3. (If you used diskettes, CD-ROM, tape, or electronic mail) Copy the packet files into a directory. 4. Apply the packet. At the receiving replica, use the syncreplica –import command to apply the changes in the packet to the replica. This example specifies the –receive option; syncreplica imports any packets it finds in the incoming shipping directories. multitool syncreplica –import –receive This example specifies a directory pathname as an argument. syncreplica –import looks in this directory for update packets and applies them to the replica on the host. multitool syncreplica –import c:\msite\packets Applied sync. packet c:\msite\packets\update1 to VOB \\servo\vobs\dev.vbs 6 - Synchronizing Replicas 103 6.6 Automated Synchronization You can use MultiSite scripts and utilities to automate all phases of synchronization: ➤ Export phase. A MultiSite export script sends update packets from one or more replicas at the site to one or more siblings. ➤ Transport phase. The store-and-forward facility handles packets of any size. You can invoke store-and-forward as part of the export phase, or automate packet transport separately. ➤ Import phase. A MultiSite receipt handler runs whenever a packet is received at a replica, and you can also schedule import of packets to occur periodically. Use scheduled jobs to automate the export and transport phases, and use receipt handlers or scheduled jobs to automate the import phase. You can run the MultiSite scripts at any time and with any frequency, and you can vary the strategy for different VOBs by using multiple jobs. By default, the MultiSite synchronization scripts place packets and shipping orders in the incoming and outgoing directories in the default storage bay, ccase-home-dir/shipping/ms_ship (UNIX) or ccase-home-dir\var\shipping\ms_ship (Windows). This bay is defined in the shipping.conf template file on UNIX and the MultiSite Control Panel on Windows. The MultiSite scripts log their activity to files in the /var/adm/atria/log/sync_logs directory on UNIX and the ccase-home-dir\var\log directory on Windows. Using the ClearCase Scheduler ClearCase installation adds three preconfigured jobs to the scheduler: Daily MultiSite Export, Daily MultiSite Shipping Poll, and Daily MultiSite Receive. These jobs use the predefined MultiSite tasks: MultiSite Sync Export and MultiSite Sync Receive. These jobs are disabled; to enable them, use the schedule –edit –schedule command or the graphical interface (Windows) and set the run times and other parameters appropriately: 104 ➤ (Using cleartool schedule) Delete the line Job.Schedule.LastDate: StartDate and set the value of Job.NotifyInfo.Recipients to the appropriate user names. ➤ (Using the scheduler graphical interface) On the Schedule tab, set the Run parameters to the appropriate values. On the Settings tab, in the Notifications section, change the value of Recipients to the appropriate user names. Administrator’s Guide: Rational ClearCase MultiSite If you decide to use receipt handlers (see Import Phase on page 107), you do not need to enable the Daily MultiSite Receive job. NOTE: For information about creating new tasks and jobs and the prerequisites for using the scheduler, see the cleartool schedule reference page in the Command Reference and the Administrator’s Guide for Rational ClearCase. Export Phase The script sync_export_list creates update packets. You can select the replicas to be updated, configure the script to send the packets immediately or place them in storage bays, and set other shipping options. For more information on the shipping options, see the sync_export_list reference page. This job runs sync_export_list to generate and send updates to all other replicas in the VOB family at midnight local time: Job.Begin Job.Id: 17 Job.Name: "Sync Export Force ALL" Job.Description.Begin: Every midnight, for each replica on this host, export update packets to all sibling replicas. Job.Description.End: Job.Schedule.Daily.Frequency: 1 Job.Schedule.FirstStartTime: 00:00:00 Job.DeleteWhenCompleted: FALSE Job.Task: 13 Job.Args: -quiet 1 -all Job.End To put the packets in a storage bay, use the –ship option. Packets in storage bays are sent by the shipping_server. For example, this job runs sync_export_list to generate an update every day at 21:00 local time: 6 - Synchronizing Replicas 105 Job.Begin Job.Id: 18 Job.Name: "Sync Export Store ALL" Job.Description.Begin: Every night at 9PM, for each replica on this host, generate update packets for all sibling replicas and store the packets in the storage bay. Job.Description.End: Job.Schedule.Daily.Frequency: 1 Job.Schedule.FirstStartTime: 21:00:00 Job.DeleteWhenCompleted: FALSE Job.Task: 13 Job.Args: -quiet 1 -ship -all Job.End See Transport Phase for information about running shipping_server. Transport Phase If sync_export_list or syncreplica puts packets in storage bays (–ship option), you must run shipping_server to process these packets. If you do not use –ship, but want to implement a retry-on-failure capability, you must schedule regular invocations of shipping_server. The shipping_server attempts to retransmit any outgoing packets that remain in any of the storage bays because one or more previous attempts have failed. With the –poll option, sync_export_list invokes shipping_server –poll to process shipping orders located in all storage bays defined in the shipping.conf file (UNIX) or in the MultiSite Control Panel (Windows). For example, this job invokes shipping_server every day at 04:00 local time: Job.Begin Job.Id: 19 Job.Name: "Shipping Server Poll" Job.Description.Begin: Every night at 4AM, run the shipping server to send any outstanding orders. Job.Description.End: Job.Schedule.Daily.Frequency: 1 Job.Schedule.FirstStartTime: 04:00:00 Job.DeleteWhenCompleted: FALSE Job.Task: 13 Job.Args: -quiet 1 -poll Job.End 106 Administrator’s Guide: Rational ClearCase MultiSite The following job implements a more aggressive retry-on-failure capability: Job.Begin Job.Id: 20 Job.Name: "Shipping Server Poll" Job.Description.Begin: Every half hour from midnight to 4AM, run the shipping server to send any outstanding orders. Job.Description.End: Job.Schedule.Daily.Frequency: 1 Job.Schedule.FirstStartTime: 00:00:00 Job.Schedule.StartTimeRestartFrequency: 00:30:00 Job.Schedule.LastStartTime: 04:00:00 Job.DeleteWhenCompleted: FALSE Job.Task: 13 Job.Args: -quiet 1 -poll Job.End Import Phase To automate packet import, use one of the methods described in Table 11. Table 11 Import Methods Import method Description Advantages Disadvantages Receipt handlers When a packet is received, the receipt handler imports it. Packets are imported immediately. Constant load on VOB server. Scheduled jobs When a packet is received at a replica, it is stored in a shipping bay. When the scheduled job runs, the packet is imported. You can schedule import to minimize server load. Changes are not applied to the VOB immediately. Receipt handlers and scheduled jobs See descriptions above. You can use scheduled jobs to implement a retry-on-failure capability. For example, if packets are delivered out of order and the receipt handler cannot import them, the job retries the import. 6 - Synchronizing Replicas 107 Defining Receipt Handlers On UNIX: You can define receipt handlers in the shipping.conf file for different shipping classes. By default, no receipt handler is defined, but you can specify the sync_receive script as a receipt handler in the shipping.conf file: RECEIPT-HANDLER -default /usr/atria/config/scheduler/tasks/sync_receive For details about defining receipt handler entries, see the section RECEIPT HANDLER in the shipping.conf reference page. On Windows: You can define receipt handlers in the MultiSite Control Panel for different shipping classes. By default, no receipt handler is defined, but you can specify ccase-home-dir\config\scheduler\tasks\sync_receive.bat in the MultiSite Control Panel. To customize sync_receive.bat, copy it to a directory outside the ClearCase installation directory, customize it, and specify it in the MultiSite Control Panel. For details about defining receipt handler entries, see the section Receipt Handler Path in the MultiSite Control Panel reference page. Scheduling Import Jobs The script sync_receive imports update packets. For more information on sync_receive options, see the sync_receive reference page. This job runs sync_receive to import all packets in the incoming shipping bays of the current host at midnight local time: Job.Begin Job.Id: 15 Job.Name: "Sync Import ALL" Job.Description.Begin: Every midnight, import all update packets in incoming bays. Job.Description.End: Job.Schedule.Daily.Frequency: 1 Job.Schedule.FirstStartTime: 00:00:00 Job.DeleteWhenCompleted: FALSE Job.Task: 14 Job.End 108 Administrator’s Guide: Rational ClearCase MultiSite 6.7 Listing Synchronization History The lshistory command and the History Browser list the history of a replica, including synchronization information. For more information, see Listing the Synchronization History of a Replica on page 113. 6.8 Synchronizing More Efficiently You can configure synchronization updates to send only the necessary operations to another replica. Although sending an operation multiple times does no harm, packet creation and transmission is more efficient if you exclude operations that have already been imported at the receiving replica. The chepoch –actual and sync_export_list –update commands contact a remote replica and update your current replica’s record of the state of the remote replica. The primary use of these commands is to resend lost packets, but you can also use them to increase synchronization efficiency. However, depending on your synchronization pattern and schedule, these commands can decrease efficiency. The following sections describe two examples: one in which efficiency is increased, and one in which it is decreased. Example of Increased Efficiency You have three replicas in a VOB family, and a subset of the synchronization pattern and schedule is shown in Figure 25. All replicas use receipt handlers, so incoming packets are imported immediately. First, replica sanfran_hub sends a packet to replica boston_hub. Next, replica boston_hub sends a packet to replica bangalore. This packet includes operations from replica sanfran_hub. 6 - Synchronizing Replicas 109 Figure 25 Partial Synchronization Export Pattern and Schedule for Three Replicas 6:00 GMT boston_hub sanfran_hub 8:00 GMT 7:00 GMT bangalore At 8:00 GMT, replica sanfran_hub sends a packet to replica bangalore. This packet contains operations originating at replica sanfran_hub that bangalore has already received from replica boston_hub. In this case, you should use the command chepoch –actual bangalore at replica sanfran_hub before generating an update packet for bangalore. When you generate the packet, the operations already imported at bangalore will be excluded from the packet. Example of Decreased Efficiency In this example, two replicas in a VOB family, sanfran_hub and sydney, exchange update packets every fifteen minutes. At some point during the day, packets may start accumulating at one of the replicas because the imports are taking a long time. For example, there is a lot of development activity in the sydney VOB and there are four packets waiting to be imported. In this case, if you run chepoch –actual at sanfran_hub before generating a packet for sydney, the update packet will contain all the operations that are already in the packets waiting to be imported at sydney. 110 Administrator’s Guide: Rational ClearCase MultiSite 7 Managing Replicas 7 This chapter describes how to manage existing replicas, including how to delete a replica. For information about creating a replica, see Chapter 4, Creating Replicas. For information about enabling requests for mastership in a replica, see Chapter 9, Implementing Requests for Mastership. 7.1 Displaying Properties of a Replica The describe command, which is available in cleartool and multitool, displays the properties of a replica. Use the –fmt option to customize the output from the command. See the fmt_ccase reference page in the Command Reference. For example, to display the name, master replica, and replica host of a replica: cleartool describe –fmt "%n\t%[master]p\t%[replica_host]p\n" \ replica:boston_hub@/vobs/dev boston_hub boston_hub@/vobs/dev minuteman You can also display properties of a replica by using a replica-uuid-selector instead of a replica-selector argument. For example (note that the replica-uuid-selector must be entered on a single line): cleartool describe –fmt "%n\n%[master]p\n%[replica_host]p\n" \ oid:87f6265f.72d911d4.a5cd.00:01:80:c0:4b:e7@replicauuid:87f6265f.72d911d4.a5cd.00:01:80:c0: 4b:e7 boston_hub boston_hub@/vobs/dev minuteman 7 - Managing Replicas 111 On Windows, the Properties Browser displays the properties of a replica. Open the Properties Browser in one of the following ways: ➤ From Windows Explorer: a. Navigate to the VOB. b. Right-click the VOB and click ClearCase > Properties of VOB. c. Click the Replicas tab. d. Select the replica and click Replica Properties. ➤ From ClearCase Administration Console: a. Navigate to All VOBs. b. Click View > List. c. Right-click the VOB and click Properties. d. Click the Replicas tab. e. Select the replica and click Replica Properties. ➤ From a command prompt: cleardescribe replica:replica-selector cleartool describe –graphical replica:replica-selector For example: cleardescribe replica:boston_hub@\dev cleartool describe –graphical replica:sanfran_hub@\tests 112 Administrator’s Guide: Rational ClearCase MultiSite 7.2 Listing the Synchronization History of a Replica The lshistory command and the History Browser (lshistory –graphical) list the synchronization history of a replica. The output differs for your current replica and its sibling replicas: ➤ When you list the history of your current replica, the output includes import events. ➤ When you list the history of a sibling replica, the output includes export events from your current replica to the sibling replica. To list the import history of your current replica (boston_hub): cleartool lshistory replica:boston_hub@/vobs/dev 17-Aug.11:05 susan import sync from replica "sanfran_hub" to replica "boston_hub" "Imported synchronization information from replica "sanfran_hub". Row at import was: boston_hub=34 sanfran_hub=0" To list all exports from your current replica to the sanfran_hub replica: cleartool lshistory replica:sanfran_hub@/vobs/dev 17-Aug.11:11 susan export sync from replica "boston_hub" to replica "sanfran_hub" "Exported synchronization information for replica "sanfran_hub". Row at export was: boston_hub=34 sanfran_hub=10" 17-Aug.10:54 susan export sync from replica "boston_hub" to replica "sanfran_hub" "Exported synchronization information for replica "sanfran_hub". Row at export was: boston_hub=0 sanfran_hub=0" 16-Aug.09:49 susan create replica "sanfran_hub" 7.3 Changing the Host Name for a Replica When you move a replica’s storage directory to a different host, or when you rename a replica’s host, you must update the host name in the replica’s VOB database. The database keeps track of the hosts on which the replicas in a VOB family reside so that the store-and-forward facility can determine how to route updates to the replicas. To change the host name, use the chreplica command or the Properties Browser (Windows). The change is not propagated to other replicas in the VOB family until you export an update packet 7 - Managing Replicas 113 from the current replica and the packet is imported at the other replicas. For restrictions, see the chreplica reference page. To change a host name using the chreplica command: multitool chreplica –host mardelplata buenosaires@/vobs/dev Updated replica information for "buenosaires". To change a host name using the Properties Browser: 1. Display properties of the replica. See Displaying Properties of a Replica on page 111. 2. On the General tab, type the new host name in the Host box. 3. Click OK or Apply. 7.4 Changing Ownership Preservation Any subset of replicas in a VOB family can be ownership-preserving. Within this group of replicas, the owner, group, and access mode of an object are kept the same across all the replicas. Adding a replica to or deleting it from the group has no immediate effect on the replica’s objects. However, future changes to object permissions are propagated among all of the ownership-preserving replicas in the VOB family. On UNIX, maintaining ownership preservation across sites is possible only if all sites support the same user-group accounts. On Windows, ownership modes (UIDs and GIDs) are not consistent across domains. Therefore, if all replicas in a VOB family are not in the same Windows domain, the entire set of replicas cannot be ownership-preserving. You can maintain ownership preservation on a subset of replicas in the same domain. In a mixed environment, you cannot maintain ownership preservation on the entire set of replicas. For more information, see Element Ownership and Ownership Preservation on page 4. NOTE: The most common change is to convert a replica from ownership-preserving to non-ownership-preserving. For example, if a replica was created incorrectly as ownership-preserving, you may need to change it. You can change a replica from non-ownership-preserving to ownership-preserving. The replica will receive future changes to ownership information, but the original ownership information is not restored. 114 Administrator’s Guide: Rational ClearCase MultiSite To change a replica’s ownership-preserving property: 1. At the master replica, change the replica property. On UNIX or Windows, you can use the chreplica command to change this property: ➣ To change from non-preserving to preserving: multitool chreplica –preserve boston_hub@/vobs/dev Updated replica information for "boston_hub". ➣ To change from preserving to non-preserving: multitool chreplica –npreserve boston_hub@/vobs/dev Updated replica information for "boston_hub". On Windows, you can use the Properties Browser to change this property: a. Display properties of the replica. See Displaying Properties of a Replica on page 111. b. To change from non-preserving to preserving, select the Ownership-preserving check box. To change from preserving to non-preserving, clear the Ownership-preserving check box. c. Click OK or Apply. See the chreplica reference page for restrictions. 2. If the changed replica is not self-mastering, export an update packet from the master replica to the changed replica. 3. At the changed replica, import the update packet. If the import fails because the VOB group lists are different, use the cleartool protectvob command to change the group list for the importing VOB replica, and then try the import again. If the import succeeds, you can use the protectvob command to delete the group you added. 4. (If the replica was changed to non-ownership-preserving) At the changed replica, use the cleartool protect command to change the ownership of all elements in the replica to the VOB owner at your site. cleartool protect –chown vobowner –chgrp vobgrp –recurse /vobs/dev 7 - Managing Replicas 115 7.5 Setting the Connectivity Property To indicate whether a sibling replica has IP connectivity to your current replica, use the chreplica command with the –isconnected or –nconnected option. This property is stored locally and is checked when you enter a command that requires connectivity to a sibling replica (for example, lsepoch –actual or chepoch –actual). The command fails quickly if the sibling replica is marked as not connected. You can also use the Properties Browser on Windows. When you display properties of a sibling replica, the General tab indicates whether the replica has IP connectivity to the current replica. You can change this property by setting or clearing the check box. To use the chreplica command to set the connectivity property to connected for the sanfran_hub replica: multitool chreplica –isconnected sanfran_hub Updated replica information for "sanfran_hub". multitool describe replica:sanfran_hub replica "sanfran-hub" ... connectivity: connected For more information, see the chreplica reference page. 7.6 Renaming a Replica To change the name of a replica, use the rename command or the Properties Browser (Windows). When you rename a replica, the change is made immediately at the current replica. The change is not propagated to other replicas in the VOB family until you export an update packet from the current replica and the packet is imported at the other replicas. You must make the change at the replica’s master replica. For other restrictions, see the rename reference page in the Command Reference. To rename a replica using the rename command: multitool rename –c "site name" replica:original@/vobs/dev replica:boston_hub@/vobs/dev Renamed replica from "original" to "boston_hub". 116 Administrator’s Guide: Rational ClearCase MultiSite To rename a replica using the Properties Browser: 1. Display properties of the replica. See Displaying Properties of a Replica on page 111. 2. Enter a new value in the Name box. 3. Click OK or Apply. 7.7 Moving a Replica See the information on moving a VOB in the Administrator’s Guide for Rational ClearCase. There are some special considerations when you move a replicated VOB: 7.8 ➤ Make sure Rational ClearCase MultiSite is installed on the new VOB server host. ➤ If you automated the synchronization process on the old host, you must set up synchronization export and import scripts on the new VOB server host. ➤ After moving the VOB replica, change the host name associated with the replica by using multitool chreplica –host. You must enter this command at the master replica of the replica you moved. ➤ After moving the VOB replica, export update packets to all sibling replicas. Changing Mastership of a Replica When you create a new replica, its replica object is mastered by the replica at which you enter the mkreplica –export command. Mastership of the replica object affects replica-modification activities (renaming the replica, changing its properties, or deleting it). You must perform these activities at the replica that masters the replica object. A self-mastering replica masters its own replica object. A replica must be self-mastering for you to perform some administrative operations on it (for example, raising the feature level). If each site has its own MultiSite administrator, having self-mastering replicas simplifies replica maintenance because each replica can be maintained at its own site. However, you may want to master all replica objects at a hub replica. In this case, you can transfer mastership to individual 7 - Managing Replicas 117 sites at the request of the site administrator, and then transfer mastership back to the hub replica after the administrative operation has been completed. To transfer mastership of a replica object: 1. Determine which replica masters the replica object, and the host name of the replica’s VOB server: multitool describe replica:sanfran_hub@/vobs/dev replica "sanfran_hub" created 16-Aug-00.09:49:36 by Susan Goechs (susan.user@minuteman) replica type: unfiltered master replica: boston_hub@/vobs/dev owner: susan group: user host: "goldengate" ... 2. At the master replica, enter a chmaster command: MINUTEMAN% multitool chmaster –c "make sanfran_hub replica self-mastering" \ sanfran_hub@/vobs/dev replica:sanfran_hub@/vobs/dev Changed mastership of replica "sanfran_hub" to "sanfran_hub@/vobs/dev" 3. At the old master replica, export an update packet to the new master replica: MINUTEMAN% multitool syncreplica –export –fship sanfran_hub@/vobs/dev Generating synchronization packet /usr/atria/shipping/ms_ship/outgoing/sync_boston_hub_16-Aug-00.16.15.57_63 89_1 - shipping order file is /usr/atria/shipping/ms_ship/outgoing/sh_o_sync_boston_hub_16-Aug-00.16.15. 57_6389_1 Attempting to forward/deliver generated packets... -- Forwarded/delivered packet /usr/atria/shipping/ms_ship/outgoing/sync_boston_hub_16-Aug-00.16.15.57_63 89_1 4. At the new master replica, import the packet: GOLDENGATE% multitool syncreplica –import –receive Applied sync. packet /usr/atria/shipping/ms_ship/incoming/sync_boston_hub_16-Aug-00.16.15.57_63 89_1 to VOB /net/goldengate/vobstg/dev.vbs 5. At the new master replica, verify that mastership has been received: 118 Administrator’s Guide: Rational ClearCase MultiSite GOLDENGATE% multitool describe replica:sanfran_hub@/vobs/dev replica "sanfran_hub" created 16-Aug-00.09:49:36 by Susan Goechs (susan.user@minuteman) replica type: unfiltered master replica: sanfran_hub@/vobs/dev ... 7.9 Deleting a Replica This section describes how to remove a replica. You must complete all steps; if you do not, synchronization and mastership problems can occur in other replicas in the VOB family. NOTE: If a VOB replica is deleted mistakenly and you want to restore it from backup, see Restoring and Replacing Replicas on page 190. If a VOB replica’s storage directory is lost and there is no backup, see Cleaning Up from Accidental Deletion of a Replica on page 196. In this scenario, the replica tokyo in the VOB family \tests is being removed. 1. At the site of the replica to be removed, complete all development work in the replica and use lscheckout or the Find Checkouts tool (Windows) to verify that all checkouts are resolved in the replica to be removed: SHINJUKU> cleartool lscheckout –all \tests (no output means no checkouts) 2. Transfer mastership of all objects to another replica. At the site of the replica to be removed, transfer mastership of all objects mastered by the replica to another replica. If the replica to be removed is not self-mastering, transfer mastership to its master replica. If the replica is self-mastering, you can choose any replica. In this example, the administrator determines which replica masters tokyo, and then transfers mastership to the master replica (in this example, sanfran_hub): SHINJUKU> cleartool describe –fmt "%[master]p\n" replica:tokyo@\tests sanfran_hub@\tests SHINJUKU> multitool chmaster –all –long sanfran_hub@\tests Changed mastership of versioned object base \tests ... Changed mastership of all objects 7 - Managing Replicas 119 The replica that receives the mastership can later transfer mastership to other replicas. If mastership is not transferred for all objects, you must fix the problem and reenter the chmaster –all –long command. For an example, see Transferring Mastership of All Objects Mastered by a Replica on page 135. If there are problems you cannot fix, another replica can recover from the error by assuming mastership of the objects. For a description of this procedure, see Cleaning Up from Accidental Deletion of a Replica on page 196. 3. Export and send an update packet from the replica to be removed. The replica to be removed must send its final changes, including checkins and mastership changes, to the replica receiving mastership. The replica to be removed can broadcast its final changes to all other replicas, but it must update its master replica (sanfran_hub in this example). SHINJUKU> multitool syncreplica –export –fship sanfran_hub@\tests 4. Import the update packet at the replica that is (or will become) the master of the replica to be removed. GOLDENGATE% multitool syncreplica –import –receive 5. At the master replica, remove the replica. GOLDENGATE% multitool rmreplica tokyo@/vobs/tests 6. At the master replica, export and send an update packet to the remaining replicas in the VOB family. This update packet notifies the other replicas of the replica removal. GOLDENGATE% multitool syncreplica –export –fship 7. At the removed replica, remove the VOB storage directory of the removed replica. SHINJUKU> cleartool rmvob \\shinjuku\vobs\tests.vbs Remove versioned object base "\\shinjuku\vobs\tests.vbs"? [no] yes Removed versioned object base "\\shinjuku\vobs\tests.vbs". If you decommission and remove all replicas, the one remaining VOB replica is a regular VOB, and developers no longer need a MultiSite license to access it. 120 Administrator’s Guide: Rational ClearCase MultiSite Managing Mastership 8 8 This chapter describes how to manage the mastership of ClearCase objects in a VOB replica, using the following commands: ➤ ➤ ➤ ➤ describe lsmaster mkelem –master chmaster The mkelem command is a cleartool subcommand. The other commands listed above are cleartool and multitool subcommands. For more information on the commands, see their reference pages in this manual or in the Command Reference. On Windows, you can use the describe and chmaster commands or the Properties Browser to display and change mastership. The reqmaster command requests mastership of branches and branch types and sets controls for mastership requests. On Windows, you can use the Request Mastership graphical interface and the Properties Browser to request mastership and set controls. Use of these interfaces is described in Chapter 9, Implementing Requests for Mastership. NOTE: Before reading this chapter, you should read the information in Enabling Independent VOB Development: Mastership on page 7. 8 - Managing Mastership 121 8.1 Listing an Object’s Master Replica To list an object’s master replica, use one of the following methods: Mastership page in the Properties Browser (Windows) cleartool describe object-selector cleartool describe –fmt "%[master]p\n" object-selector This page lists the object’s master replica. This command lists general information about the object, including its master replica. This command lists only the master replica of the object. For more information on using the –fmt option, see the fmt_ccase reference page in the Command Reference. Command examples: ➤ To list a replica object’s master replica: cleartool describe replica:boston_hub@\dev replica "boston_hub" created 15-Aug-00.14:19:03 by susan.user replica type: unfiltered master replica: boston_hub@\dev ... ➤ To list an element’s master replica: cleartool describe –fmt "%n\t%[master]p\n" cmdsyn.c@@ cmdsyn.c@@ ➤ bangalore@/vobs/dev To list a type object’s master replica: cleartool describe lbtype:V1.0@/vobs/dev label type "V1.0" created 25-Aug-00.12:14:52 by Susan Goechs (susan.user@minuteman) master replica: boston_hub@/vobs/dev ... ➤ To list a branch’s master replica: cleartool describe –fmt "%n\t%[master]p\n" cmdsyn.c@@\main\v3_bugfix cmdsyn.c@@\main\v3_bugfix boston_hub@\dev 122 Administrator’s Guide: Rational ClearCase MultiSite 8.2 Listing Objects Mastered by a Replica The lsmaster command lists the objects mastered by a replica. The command uses the information at your current replica unless you use the –inreplicas option, which retrieves information from sibling replicas. ➤ To list all objects mastered by the current replica (boston_hub): multitool lsmaster boston_hub@/vobs/dev ➤ To list all label types mastered by replica sanfran_hub, using information in the current replica: multitool lsmaster –kind lbtype sanfran_hub@/vobs/dev ➤ To display information from all replicas in the VOB family about the objects mastered by replica bangalore: multitool lsmaster –inreplicas –all bangalore@\tests For more information and examples, see the lsmaster reference page. 8.3 Listing the History of Mastership Changes for an Object The lshistory –minor command and the History Browser (with the Include minor events toggle selected) list mastership changes for an object. For example, to list the history of a replica: cleartool lshistory –minor replica:bangalore@/vobs/dev 20-Sep.17:43 susan change master to "bangalore" of replica "bangalore" "Changed master replica from "boston_hub" to "bangalore"." 8 - Managing Mastership 123 8.4 Displaying Mastership Request Settings The mastership request setting controls whether developers at other sites can request mastership of branches or branch types mastered by the replica. The describe command and (on Windows) the Mastership page in the Properties Browser display mastership request settings for replicas, branch types, and branches. For more information on mastership requests, see Chapter 9, Implementing Requests for Mastership. 8.5 Assigning Branch Mastership During Element Creation By default, when you create an element in a replicated VOB, mastership of the branch main is assigned to the replica that masters the branch type main. If this replica is not your current replica, you cannot create new versions on the main branch. Also, if your config spec contains mkbranch rules and your current replica does not master the branch types, the branches cannot be created during element creation. To assign mastership of a new element’s main branch, and all other branches created during element creation, to your current replica, do one of the following: ➤ Use the command cleartool mkelem –master. ➤ (Windows) In the Add to Source Control dialog box, select Make current replica the master of all newly created branches. For example, suppose your view has the following config spec: element * CHECKEDOUT element * .../v1.0_bugfix/LATEST element * V1.0 -mkbranch v1.0_bugfix Use the following procedure to assign mastership of new branches to your current replica: 1. Create a new element with mkelem –master and check out the file: cleartool mkelem –master –c "adding comments" cmdsyn.c Created element "cmdsyn.c" (type "text_file"). Created branch "v1.0_bugfix" from "cmdsyn.c" version "/main/0". Note: Branch "v1.0_bugfix" explicitly mastered by replica "boston_hub". Branch type "v1.0_bugfix" mastered by replica "sanfran_hub". Checked out "cmdsyn.c" from version "/main/v1.0_bugfix/0". 124 Administrator’s Guide: Rational ClearCase MultiSite 2. Use the describe command to confirm that the new branches are mastered by your current replica: cleartool describe cmdsyn.c@@/main cmdsyn.c@@/main/v1.0_bugfix branch "cmdsyn.c@@/main" created 02-Sep-00.13:17:21 by Gail Smith (gail.user@boston30) branch type: main master replica: boston_hub@/vobs/dev ... branch "cmdsyn.c@@/main/v1.0_bugfix" created 02-Dec-00.13:17:21 by Gail Smith (gail.user@boston30) branch type: v1.0_bugfix master replica: boston_hub@/vobs/dev ... If you make your current replica the master of newly created branches, but do not check out the file (that is, you use the –nco option), only the main branch is mastered by your current replica, because it is the only branch that is created. For example: 1. Create a new element with mkelem –nco –master: cleartool mkelem –nco –master –c "adding comments" cmdsyn.c cleartool: Warning: Moved private data from "cmdsyn.c" to "cmdsyn.c.keep" so it won’t eclipse element. Created element "cmdsyn.c" (type "text_file"). 2. Use the describe command to confirm that the main branch is mastered by your current replica: cleartool describe cmdsyn.c@@/main branch "cmdsyn.c@@/main" created 02-Sep-00.13:21:21 by Gail Smith (gail.user@boston30) branch type: main master replica: boston_hub@/vobs/dev ... 3. List the element’s history to confirm that no other branches except main were created: cleartool lshistory cmdsyn.c 02-Sep.13:21 02-Sep.13:21 02-Sep.13:21 8 - Managing Mastership gail gail gail create version "cmdsyn.c@@/main/0" create branch "cmdsyn.c@@/main" create file element "cmdsyn.c@@" 125 8.6 Changing Mastership When you create an object in a replicated VOB, your current replica is the new object’s master. You can transfer mastership of the object to another replica, using the chmaster command or the Properties Browser (Windows). Some examples of when this is appropriate: ➤ If responsibility for product integration is shifted to a different site, you must transfer mastership of the integration branch types or branches to the new site. ➤ Before you decommission a replica, you must transfer mastership of each object mastered by that replica to one of the remaining replicas. (See Deleting a Replica on page 119.) Mastership changes are communicated among replicas by the standard synchronization mechanism. The general procedure for changing mastership is as follows: 1. Change mastership of one or more objects to another replica or request mastership of a branch or branch type. 2. Export and send an update packet from the old master replica to the new master replica. (The reqmaster command does this automatically.) 3. Import the update packet at the new master replica. Until the update packet containing the mastership change is imported at the new master replica, mastership is “in the packet” and the replicas in the VOB family have different information about which replica masters the object. For example, the administrator at the sanfran_hub replica transfers mastership of the bugfix branch to the bangalore replica, and then exports an update packet. At this point: ➤ ➤ ➤ The sanfran_hub replica considers the branch to be mastered by bangalore. The bangalore replica considers the branch to be mastered by sanfran_hub. No one can create new versions on the branch. When you complete the mastership transfer by importing the update packet at bangalore, developers at bangalore are able to create new versions on the branch. Notes on mastership changes: 126 ➤ The chmaster command requires a view context. ➤ If your VOB family includes any read-only or one-way replicas (replicas that import update packets but do not export them), be careful about transferring mastership to these replicas. Administrator’s Guide: Rational ClearCase MultiSite After you give mastership of an object to a read-only or one-way replica, you cannot change the object’s mastership again unless you change the VOB family’s synchronization pattern. ➤ You cannot undo a mastership change made at your site by making the opposite change at your site. See Fixing an Accidental Mastership Change on page 137. ➤ You can use triggers to prevent developers from changing mastership of objects. For more information, see Implementing Project Development Policies in Managing Software Projects. The following sections describe how to change mastership of ClearCase objects. These procedures use the command line. For information about using the Properties Browser on Windows to transfer mastership of a ClearCase object, see the MultiSite online help: 1. Click Start > Programs > Rational ClearCase Administration > MultiSite Help. 2. On the Contents tab of the Help Contents Window, click Administrator Tasks > To change mastership of a ClearCase object. Transferring Mastership of a Type Object When you create a type object, it is mastered by the replica where you create it. Except for elements, instances of an unshared type can be created only at the master replica. Elements can be created at any replica, regardless of which replica masters the element type. Instances of shared types can be created at any replica (for more information, see Type Object Mastership on page 14). If you transfer mastership of a branch type to another replica, mastership of explicitly mastered branches of that type is not transferred, even if the same replica masters the branch type and the branch. To give such branches default mastership, see the procedure in Removing Explicit Mastership of a Branch on page 133. NOTE: To transfer mastership of a type object: 1. Determine which replica masters the type object: multitool describe lbtype:TOKYO_BASE@/vobs/dev label type "TOKYO_BASE" created 15-Aug-00.14:20:26 by Susan Goechs (susan.user@minuteman) master replica: boston_hub@/vobs/dev ... 8 - Managing Mastership 127 2. At the master replica, enter a chmaster command: MINUTEMAN% multitool chmaster –c "transfer to sanfran_hub" \ sanfran_hub@/vobs/dev lbtype:TOKYO_BASE@/vobs/dev Changed mastership of label type "TOKYO_BASE" to "sanfran_hub@/vobs/dev" 3. At the old master replica, export an update packet to the new master replica’s site: MINUTEMAN% multitool syncreplica –export –fship sanfran_hub@/vobs/dev Generating synchronization packet /usr/atria/shipping/ms_ship/outgoing/sync_boston_hub_26-Aug-00.18.15.57_74 30_1 - shipping order file is /usr/atria/shipping/ms_ship/outgoing/sh_o_sync_boston_hub_26-Aug-00.18.15. 57_7430_1 Attempting to forward/deliver generated packets... -- Forwarded/delivered packet /usr/atria/shipping/ms_ship/outgoing/sync_boston_hub_26-Aug-00.18.15.57_74 30_1 4. At the new master replica, import the packet: BAGUETTE% multitool syncreplica –import –receive Applied sync. packet /usr/atria/shipping/ms_ship/incoming/sync_boston_hub_26-Aug-00.18.15.57_74 30_1 to VOB /net/goldengate/vobstg/dev.vbs 5. At the new master replica, verify that mastership has been received: BAGUETTE% multitool describe lbtype:TOKYO_BASE@/vobs/dev label type "TOKYO_BASE" created 15-Aug-00.14:20:26 by Susan Goechs (susan.user@minuteman) master replica: sanfran_hub@/vobs/dev ... Transferring Mastership of a Replica Object When you create a new replica, its replica object is mastered by the replica at which you enter the mkreplica –export command. Mastership of the replica object affects replica-modification activities (renaming the replica, changing its properties, or deleting it). You must perform these activities at the replica that masters the replica object. 128 Administrator’s Guide: Rational ClearCase MultiSite A self-mastering replica masters its own replica object. A replica must be self-mastering for you to perform some administrative operations on it (for example, raising the feature level). If each site has its own MultiSite administrator, having self-mastering replicas simplifies replica maintenance because each replica can be maintained at its own site. However, you may want to master all replica objects at a hub replica. In this case, you can transfer mastership to individual sites at the request of the site administrator, and then transfer mastership back to the hub replica after the administrative operation has been completed. To transfer mastership of a replica object: 1. Determine which replica masters the replica object, and the host name of the replica’s VOB server: multitool describe replica:sanfran_hub@/vobs/dev replica "sanfran_hub" created 16-Aug-00.09:49:36 by Susan Goechs (susan.user@minuteman) replica type: unfiltered master replica: boston_hub@/vobs/dev owner: susan group: user host: "goldengate" ... 2. At the master replica, enter a chmaster command: MINUTEMAN% multitool chmaster –c "make sanfran_hub replica self-mastering" \ sanfran_hub@/vobs/dev replica:sanfran_hub@/vobs/dev Changed mastership of replica "sanfran_hub" to "sanfran_hub@/vobs/dev" 3. At the old master replica, export an update packet to the new master replica: MINUTEMAN% multitool syncreplica –export –fship sanfran_hub@/vobs/dev Generating synchronization packet /usr/atria/shipping/ms_ship/outgoing/sync_boston_hub_16-Aug-00.16.15.57_63 89_1 - shipping order file is /usr/atria/shipping/ms_ship/outgoing/sh_o_sync_boston_hub_16-Aug-00.16.15. 57_6389_1 Attempting to forward/deliver generated packets... -- Forwarded/delivered packet /usr/atria/shipping/ms_ship/outgoing/sync_boston_hub_16-Aug-00.16.15.57_63 89_1 4. At the new master replica, import the packet: 8 - Managing Mastership 129 GOLDENGATE% multitool syncreplica –import –receive Applied sync. packet /usr/atria/shipping/ms_ship/incoming/sync_boston_hub_16-Aug-00.16.15.57_63 89_1 to VOB /net/goldengate/vobstg/dev.vbs 5. At the new master replica, verify that mastership has been received: GOLDENGATE% multitool describe replica:sanfran_hub@/vobs/dev replica "sanfran_hub" created 16-Aug-00.09:49:36 by Susan Goechs (susan.user@minuteman) replica type: unfiltered master replica: sanfran_hub@/vobs/dev ... Transferring Mastership of a VOB When you replicate a VOB for the first time, the VOB is mastered by the original replica. You must perform the following operations at the VOB’s master replica: ➤ ➤ Changing protections on the VOB (for ownership-preserving replicas). Locking the VOB with the obsolete option. To transfer mastership of a VOB to another replica, follow these steps: 1. At the master replica, enter a chmaster command: MINUTEMAN% multitool chmaster sanfran_hub vob:/vobs/dev Changed mastership of versioned object base "/vobs/dev" to "sanfran_hub". 2. At the old master replica, export an update packet to the new master replica’s site: MINUTEMAN% multitool syncreplica –export –fship sanfran_hub@/vobs/dev Generating synchronization packet /usr/atria/shipping/ms_ship/outgoing/sync_boston_hub_20-Sep-00.17.35.45_22 513_1 - shipping order file is /usr/atria/shipping/ms_ship/outgoing/sh_o_sync_boston_hub_20-Sep-00.17.35. 45_22513_1 Attempting to forward/deliver generated packets... -- Forwarded/delivered packet /usr/atria/shipping/ms_ship/outgoing/sync_boston_hub_20-Sep-00.17.35.45_22 513_1 130 Administrator’s Guide: Rational ClearCase MultiSite 3. At the new master replica, import the packet: GOLDENGATE% multitool syncreplica –import –receive Applied sync. packet /usr/atria/shipping/ms_ship/incoming/sync_boston_hub_20-Sep-00.17.35.45_22 513_1 to VOB /net/goldengate/vobstg/dev.vbs 4. At the new master replica, verify that mastership has been received: GOLDENGATE% multitool describe –fmt "%n\t%[master]p\n" vob:/vobs/dev /vobs/dev sanfran_hub@/vobs/dev Transferring Mastership of an Element When you create a new element, it is mastered by the replica in which you create it. You must perform the following element operations at the element’s master replica: ➤ ➤ ➤ Changing protections on the element (for ownership-preserving replicas). Locking the element with the obsolete option. Removing the element. To transfer mastership of an element to another replica, follow these steps: 1. At the master replica, enter a chmaster command: MINUTEMAN% multitool chmaster bangalore tests.txt@@ Changed mastership of file element "tests.txt@@" to "bangalore" 2. At the old master replica, export an update packet to the new master replica’s site: MINUTEMAN% multitool syncreplica –export –fship bangalore@/vobs/dev Generating synchronization packet /usr/atria/shipping/ms_ship/outgoing/sync_boston_hub_07-Dec-00.18.15.57_59 78_1 - shipping order file is /usr/atria/shipping/ms_ship/outgoing/sh_o_sync_boston_hub_07-Dec-00.18.15. 57_5978_1 Attempting to forward/deliver generated packets... -- Forwarded/delivered packet /usr/atria/shipping/ms_ship/outgoing/sync_boston_hub_07-Dec-00.18.15.57_59 78_1 8 - Managing Mastership 131 3. At the new master replica, import the packet: RAMOHALLI> multitool syncreplica –import –receive Applied sync. packet C:\Program Files\Rational\ClearCase\var\shipping\ms_ship\incoming\sync_boston_hub_07Dec-00.18.15.57_5978_1 to VOB \\ramohalli\vobs\dev.vbs 4. At the new master replica, verify that mastership has been received: RAMOHALLI> multitool describe –fmt "%n\t%[master]p\n" tests.txt@@ tests.txt@@ bangalore@\dev Transferring Mastership of a Branch This section describes how to change mastership of a branch using the chmaster command. For information about enabling use of the reqmaster command, see Chapter 9, Implementing Requests for Mastership. Transferring Branch Mastership To transfer mastership of a branch to another replica: 1. At the master replica, enter a chmaster command: MINUTEMAN% multitool chmaster –c "bugfix at bangalore" bangalore Makefile@@/main Changed mastership of branch "Makefile@@/main" to "bangalore" 2. At the old master replica, export an update packet to the new master replica’s site: MINUTEMAN% multitool syncreplica –export –fship bangalore@/vobs/dev Generating synchronization packet /usr/atria/shipping/ms_ship/outgoing/sync_boston_hub_10-Dec-00.18.15.57_30 56_1 - shipping order file is /usr/atria/shipping/ms_ship/outgoing/sh_o_sync_boston_hub_10-Dec-00.18.15. 57_3056_1 Attempting to forward/deliver generated packets... -- Forwarded/delivered packet /usr/atria/shipping/ms_ship/outgoing/sync_boston_hub_10-Dec-00.18.15.57_30 56_1 132 Administrator’s Guide: Rational ClearCase MultiSite 3. At the new master replica, import the packet: RAMOHALLI> multitool syncreplica –import –receive Applied sync. packet C:\Program Files\Rational\ClearCase\var\shipping \ms_ship\incoming\sync_boston_hub_10-Dec-00.18.15.57_3056_1 to VOB \\ramohalli\vobs\dev.vbs 4. At the new master replica, verify that mastership has been received: RAMOHALLI> multitool describe –fmt "%n\t%[master]p\n" Makefile@@\main Makefile@@\main bangalore@\dev Removing Explicit Mastership of a Branch As described in Default and Explicit Branch Mastership on page 13, a branch can have default or explicit mastership. After you follow the steps in Transferring Branch Mastership on page 132, the branch has explicit mastership. When you transfer mastership of a branch type to another replica, mastership is transferred for all branches with default mastership, but not for branches with explicit mastership. To return mastership of a branch to the replica that masters the branch type: 1. At the replica that masters the branch, enter a chmaster –default command: RAMOHALLI> multitool chmaster –default Makefile@@\main Changed mastership of branch "Makefile@@\main" to "default" 2. Determine which replica masters the branch type: RAMOHALLI> multitool describe –fmt "%n\t%[master]p\n" brtype:main main boston_hub@\dev If your current replica masters the branch type, stop here. If another replica masters the branch type, continue with Step #3. 8 - Managing Mastership 133 3. Export an update packet to the replica that masters the branch type: RAMOHALLI> multitool syncreplica –export –fship boston_hub@\dev Generating synchronization packet C:\Program Files\Rational\ClearCase\var\shipping\ms_ship\outgoing\sync_bangalore_11-D ec-00.18.15.57_9476_1 - shipping order file is /usr/atria/shipping/ms_ship/outgoing/sh_o_sync_bangalore_11-Dec-00.18.15.5 7_9476_1 Attempting to forward/deliver generated packets... -- Forwarded/delivered packet /usr/atria/shipping/ms_ship/outgoing/sync_bangalore_11-Dec-00.18.15.57_947 6_1 4. At the replica that masters the branch type, import the packet: MINUTEMAN% multitool syncreplica –import –receive Applied sync. packet /usr/atria/shipping/ms_ship/incoming/sync_bangalore_11-Dec-00.18.15.57_947 6_1 to VOB /net/minuteman/vobstg/dev.vbs 5. At the replica that masters the branch type, verify that the branch has default mastership: MINUTEMAN% multitool describe Makefile@@/main branch "Makefile@@/main" created 27-Aug-00.13:41:21 by Gail Smith (gail.user@boston20) branch type: main master replica: boston_hub@/vobs/dev (defaulted) The other form of the chmaster –default command applies to branches that are explicitly mastered by the replica that masters the branch type. To give these branches default mastership, enter a chmaster –default command and specify the branch type: MINUTEMAN% multitool chmaster –default brtype:main Changed mastership of branch type "main" to "default" 134 Administrator’s Guide: Rational ClearCase MultiSite Transferring Mastership of a Stream The chmaster –stream command transfers mastership of a stream and its associated objects. For example, to transfer mastership of the stream v2.1.bl5 and its associated objects to the boston_hub replica: multitool chmaster –stream boston_hub@/vobs/dev stream:v2.1.bl5@/vobs/dev In some cases, you must manually change mastership of branch types or activities associated with a stream. The output of the chmaster command includes a list of these objects. The output may also include an instruction to run the chmaster –stream command with the –override option. This option transfers mastership of objects whose mastership was not transferred during the original invocation of the command. CAUTION: Do not use –override unless the output of chmaster –stream indicates that you should do so. Transferring Mastership of All Objects Mastered by a Replica Before removing a replica, you must transfer mastership of all objects mastered by that replica. For detailed instructions, see Deleting a Replica on page 119. The following example shows a partially successful chmaster –all command and describes how to fix it. In this example, the administrator at replica bangalore is transferring mastership to boston_hub. RAMOHALLI> multitool chmaster –all –long boston_hub@\dev Changed mastership of versioned object base \dev\ Changed mastership of directory element \dev\.@@ Changed mastership of directory element \dev\lost+found@@ ... multitool: Error: Branch type "bangalore_main" has branches (with default mastership) that have outstanding checkouts. Changed mastership of branch type v1.0_bugfix ... multitool: Error: Lock on label type "V1.0_BUGFIX" prevents operation "change master". Changed mastership of label type BANGALORE_V2.0 ... Changed mastership of replica bangalore multitool: Warning: Not all objects had mastership changed. 8 - Managing Mastership 135 These errors prevent the successful completion of this chmaster command: ➤ ➤ There are checkouts on the bangalore_main branch. There is a lock on a label type. To fix these problems: 1. Find the checkouts and either check in the files or cancel the checkouts: H:\dev> cleartool lscheckout –all 03-Jun.17:28 jk checkout version "\dev\cmdsyn.c" from \main\bangalore_main\83 (unreserved) 08-Jun.12:45 singh checkout version "\dev\etc\util\tool.c" from \main\bangalore_main\22 (unreserved) ... See the checkin, checkout and uncheckout reference pages. 2. Unlock the type object. cleartool unlock lbtype:V1.0_BUGFIX@\dev Unlocked label type "V1.0_BUGFIX". Alternatively, enter a lock –replace –nusers command and add yourself to the –nusers list. cleartool lock –replace –nusers ms_admin lbtype:V1.0_BUGFIX@\dev Locked label type "V1.0_BUGFIX". 3. Reenter the chmaster command. RAMOHALLI> multitool chmaster –all –long boston_hub@\dev Changed mastership of branch type bangalore_main Changed mastership of label type V1.0_BUGFIX Changed mastership of all objects. 136 Administrator’s Guide: Rational ClearCase MultiSite Fixing an Accidental Mastership Change If a mastership change is made in your replica by mistake, follow these steps to undo the change: 1. At your replica, complete the transfer by sending an update packet to the new master replica. 2. At the new master replica, complete these steps: a. Import the packet. b. Change mastership back to your replica. c. Export an update packet to your replica. 3. At your replica, import the packet. 8.7 Working with Type Objects When you create an attribute type, a hyperlink type, or a label type, you can make the type shared or unshared. By default, the type is unshared, which means that instances of the type can be created only at the replica that masters the type object. If you define the type object to be shared, instances of the type can be created at any replica in the VOB family. For more information about type objects, see Type Object Mastership on page 14. Creating a Shared Type Object To create a shared type object, use the –shared option with the mkattype, mkhltype, or mklbtype command. For example, to create a shared attribute type: cleartool mkattype –shared –c "testing status" TESTED Created attribute type "TESTED". 8 - Managing Mastership 137 Listing Whether a Type Object Is Shared or Unshared On Windows, the Properties Browser displays the kind of mastership on the Mastership tab. The describe command includes the kind of mastership in its output: cleartool describe attype:TESTED attribute type "TESTED" created 15-Aug-00.14:23:27 by Susan Goechs (susan.user@minuteman) master replica: boston_hub@/vobs/dev instance mastership: shared ... You can also use the –fmt option to display the kind of mastership. For example, to list the mastership kind of a single type object: cleartool describe –fmt "%n\t%[type_mastership]p\n" attype:TESTED TESTED shared To list the mastership kind of all label types in a VOB replica: cleartool lstype –fmt "%n\t%[type_mastership]p\n" –kind lbtype BACKSTOP shared BANGALORE_BASE unshared BUENOSAIRES_BASE unshared CHECKEDOUT shared LATEST shared BOSTON_BASE unshared SANFRAN_BASE unshared V1.0 unshared V2.0 unshared Converting an Unshared Type Object to a Shared Type Object You can convert an unshared attribute type, hyperlink type, or label type to be shared. For example, if a project manager at the San Francisco site creates an unshared attribute type called TESTED_BY, but the Bangalore project manager also needs to use this type, you can convert the type to shared so both project managers can create instances of the type. NOTE: You cannot convert a shared type object to unshared. To restrict instance creation of a type to one site, remove all instances of the type, remove the type, and create a new unshared type. 138 Administrator’s Guide: Rational ClearCase MultiSite For information about using the Properties Browser on Windows to convert an unshared type object to a shared type object, see the MultiSite online help: 1. Click Start > Programs > Rational ClearCase Administration > MultiSite Help. 2. On the Contents tab of the Help Contents Window, click Administrator Tasks > To change a type to have shared mastership. To use the command line to convert an unshared type object to a shared type object: 1. Determine which replica masters the type object: cleartool describe attype:TESTED_BY@/vobs/stage attribute type "TESTED_BY" created 03-Oct-00.10:29:06 by John Cole (jcole.user@goldengate) master replica: sanfran_hub@/vobs/dev instance mastership: unshared ... 2. At the master replica, enter a mk**type –replace –shared command to replace the definition of the type: cleartool mkattype –replace –shared –c "needed at multiple sites" TESTED_BY Replaced definition of attribute type "TESTED_BY". 3. Export an update packet to the other sites that must use the type: multitool syncreplica –export –fship bangalore boston_hub ... 4. At the receiving sites, import the update packet: multitool syncreplica –import –receive ... 5. At the receiving sites, confirm that the type object is shared: cleartool describe –fmt "%n\t%[type_mastership]p\n" \ attype:TESTED_BY@/vobs/dev TESTED_BY 8 - Managing Mastership shared 139 140 Administrator’s Guide: Rational ClearCase MultiSite 9 Implementing Requests for Mastership 9 To support development of elements that cannot be merged, you can give developers the ability to request mastership of branches and branch types. This chapter describes how these requests work, the requirements and recommendations for enabling requests, the planning you must do, and the procedure for enabling requests. Before reading this chapter, read the information on branch mastership in Chapter 1, Introduction to MultiSite, and Branching and Mastership on page 36. 9.1 Overview of a Request for Mastership When a developer requests mastership of a branch, the branch’s mastership is transferred to the developer’s current replica. When a developer requests mastership of a branch type, mastership of the branch type, along with mastership of all the instances of the branch type that have default mastership, is transferred to the developer’s current replica. The procedure for requesting mastership is as follows: 1. A developer makes a request for mastership. 2. The developer’s client host determines which replica masters the branch or branch type, and sends a request for mastership to that replica. This request is made directly to the VOB server, not by sending an update packet. 3. Authorization checking occurs at the sibling replica. The checks are different for a branch and a branch type. 9 - Implementing Requests for Mastership 141 For a request for mastership of a branch, authorization checking determines the following: a. Whether the developer is allowed to request mastership. b. Whether requests for mastership of the branch are allowed at the replica level, the branch type level, and the branch level. c. Whether the replica masters the branch. If the replica does not master the branch, the mastership request fails. The process in Step #2 uses the information available from the client host’s current replica. If the sibling replica has transferred mastership of the branch to another replica, but the current replica has not received an update packet with the change, the information at the current replica is not up to date. d. Whether the branch, its branch type, or VOB is locked. If one or more of these objects are locked, the request fails. e. Whether there are any checkouts on the branch, except for nonmastered checkouts. A reserved or unreserved checkout on the branch causes the request to fail. f. Whether the branch is associated with a stream. You cannot request mastership of a branch associated with a stream. For a request for mastership of a branch type, authorization checking determines the following: a. Whether the developer is allowed to request mastership. b. Whether requests for mastership of the branch type are allowed at the replica level and the branch type level. Also, whether requests are allowed for any of the branch type’s instances that have default mastership. c. Whether the replica masters the branch type. If the replica does not master the branch type, the mastership request fails. The process in Step #2 uses the information available from the client host’s current replica. If the sibling replica has transferred mastership of the branch type to another replica, but the current replica has not received an update packet with the change, the information at the current replica is not up to date. d. Whether any of the following objects are locked: the branch type, the VOB, or any of the branch type’s instances that have default mastership. If one or more of these objects are locked, the request fails. 142 Administrator’s Guide: Rational ClearCase MultiSite e. Whether there are any checkouts (except for nonmastered checkouts) on any of the branch type’s instances that have default mastership. f. Whether the branch type is associated with a stream. You cannot request mastership of a branch type associated with a stream. If the request passes the authorization checks, the process continues with Step #4. (If the developer requests mastership of multiple branches or branch types, error messages are printed for the failures and processing continues.) 4. The server process for the sibling replica assigns mastership of the branch or branch type to the developer’s current replica. The event record for this operation includes the user name of the requesting developer as part of the comment. At this point, the sibling replica is the only replica in the VOB family that has information about the mastership change. At all other replicas in the family, including the developer’s current replica, the current mastership information shows that the sibling replica masters the branch or branch type. The developer’s current replica is updated when the packet created in Step #5 is imported. The other replicas in the family are not updated until they are synchronized with either of the two replicas that has information about the change. 5. The server at the sibling replica starts an export process to create and send an update packet containing the mastership change to the developer’s current replica. This packet also contains other changes made since the last synchronization export. 6. The mastership request operation completes its processing. After the update packet is imported successfully at the developer’s current replica, the branch or branch type is mastered by the current replica and developers at the site can create new versions on the branch or create new instances of the branch type. NOTE: A request for mastership does not initiate a syncreplica –import command. If the replica’s host uses a receipt handler (the recommended procedure), the import begins as soon as the packet arrives. Otherwise, the import occurs at the scheduled import time at the site or when an administrator imports the packet manually. 9 - Implementing Requests for Mastership 143 9.2 Requirements and Recommendations To enable requests for mastership in one or more replicas, the following conditions must apply: ➤ The VOB family is at feature level 2 or higher. (All replicas in the VOB family must be at feature level 2 or higher, even if you are not going to enable requests in all of the replicas.) For more information on feature levels, see Chapter 5, ClearCase Feature Levels. ➤ The sites have high-speed connections (LAN, WAN, T1). A request for mastership makes RPCs directly to remote servers and fails if the sites are not connected. If a site has a firewall, developers at that site cannot request mastership from replicas at other sites, and developers at other sites cannot request mastership of any branches mastered at a site with a firewall. ➤ Each replica masters its own replica object. These replicas are called self-mastering. If a replica does not master its own replica object, you cannot enable or disable mastership requests at the replica level. For information about reassigning mastership of the replica object, see Transferring Mastership of a Replica Object on page 128. For mastership requests to work efficiently, the following conditions must apply: ➤ There is no contention for branches or branch types among the sites. That is, only one person at a time requests mastership of a branch or branch type. If two or more developers at different sites compete for mastership of objects, mastership will always be in flux. In this situation, the project leaders and MultiSite administrators must determine whether the branch sharing strategy needs to be changed. Using requests for mastership is not a substitute for using good branching and merging practices. ➤ The sites exchange update packets frequently. Each replica needs current information about object mastership. If a replica is not up to date, requests for mastership from that site cannot determine which replica masters the requested object. Also, if replicas exchange packets infrequently, a mastership request may cause the generation of a large update packet, which will take longer to generate and import.) ➤ Each replica host uses a receipt handler to import packets. You can schedule scripts to import packets regularly. However, to import a packet as soon as it arrives at the replica host, you must use a receipt handler. For more information, see the shipping.conf (UNIX) or MultiSite Control Panel (Windows) reference page. 144 Administrator’s Guide: Rational ClearCase MultiSite 9.3 Planning Your Implementation Before enabling requests for mastership, the project managers and administrators at the different sites must make these decisions: ➤ Which replicas must be enabled to allow requests for mastership. By default, a replica does not allow requests for mastership. You can enable one replica, multiple replicas, or all replicas in a VOB family. ➤ Which developers are authorized to request mastership. By default, no one is authorized. You can authorize individual developers, everyone in a specific group, everyone in a specific domain, or everyone in your network. ➤ The branch types and branches (if any) for which mastership requests are always denied. By default, requests are allowed. Although you can enable requests for mastership in components, you cannot request mastership of a branch or branch type associated with a stream. To Hide Request for Mastership Features If you do not implement requests for mastership at particular sites, you can hide request for mastership features in the ClearCase graphical interface on Windows. The display of these features is controlled by the site-wide setting rfm_gui_visibility. To use the setsite command to hide request for mastership features: cleartool setsite rfm_gui_visibility=FALSE To use ClearCase Administration Console to hide request for mastership features: 1. Navigate to the ClearCase Registry node in ClearCase Administration Console. 2. Click Action > Properties. 3. Click Help and follow the instructions in the online help. 9 - Implementing Requests for Mastership 145 9.4 Enabling Requests for Mastership The procedures in this section use the command line. On Windows, you can use the ACL editor and the Properties Browser. For more information, see the MultiSite online help: 1. Click Start > Programs > Rational ClearCase Administration > MultiSite Help. 2. On the Contents tab of the Help Contents Window, click Administrator Tasks > Enabling Requests for Mastership > To enable requests for mastership. Prerequisites 1. Ensure that the replica is self-mastering. See Transferring Mastership of a Replica Object on page 128. 2. Ensure that the feature level of the replicas in the VOB family is the correct value, and that the VOB family’s feature level is the correct value. For instructions, see Chapter 5, ClearCase Feature Levels. Adding Developers to the Access Control List 3. At each replica, add the appropriate people to the replica’s access control list. multitool reqmaster –acl –edit vob-selector A replica’s access control list (ACL) contains a list of users at other sites who are allowed to request mastership of branches and branch types mastered by that replica. To modify this file, you must be VOB owner, root (on UNIX), a member of the ClearCase administrators group (on Windows), or have write permissions on the ACL. The vob-selector specifies a VOB family, and the ACL for your current replica is changed. An access control list contains lines of the following form: identity-specification access-level,... identity-specification is one of the following: Everyone Domain:domain 146 Everyone in all domains. Everyone in the specified domain. Administrator’s Guide: Rational ClearCase MultiSite Group:domain/group User:domain/username Everyone in the specified group in domain. You can use a slash ( / ) or backslash ( \ ) between domain and group. A specific user in a particular domain. You can use a slash ( / ) or backslash ( \ ) between domain and username. On Windows, domain is the name of a Windows domain (for example, purpledoc). On UNIX, domain is an NIS domain name (for example, purpledoc.com). If someone who can request mastership has user names in multiple domains, you must specify all the identities in the ACL. access-level is one or more of the following: Read Write Change Full Allow read access on ACL Allow write access on ACL Allow requests for mastership Allow requests for mastership and read/write access on ACL Separate multiple access levels with a comma, but do not include spaces between access levels. The identity specification and associated access levels must appear on the same line. For example, the following ACL specifies that susan can modify the ACL, and jcole and kumar can request mastership: User:purpledoc.com/susan Read,Write User:purpledoc/susan Read,Write User:purpledoc.com/jcole Change User:purpledoc/jcole Change User:purpledoc.com/kumar Change User:purpledoc/kumar Change The following ACL gives msadm full permissions and allows everyone to request mastership: User:purpledoc.com/msadm Full User:purpledoc/msadm Full Everyone Change 9 - Implementing Requests for Mastership 147 Deny Requests for Specific Objects 4. (optional) At each replica, deny requests for mastership of specific objects. By default, requests are allowed for all branches and branch types. multitool reqmaster –deny branch-pname multitool reqmaster –deny branch-type-selector multitool reqmaster –deny –instances branch-type-selector Denies requests for mastership of the specified branch. Denies requests for mastership of the specified branch type. Denies requests for mastership of all instances of the specified branch type. For you to allow or deny mastership requests for a branch or branch type, your current replica must master it. You can allow or deny mastership requests for all instances of a branch type even if your current replica does not master the type. If the branch type is a global type, its mastership request setting is stored in the administrative VOB and applies to all local copies of the branch type. Enable Requests at the Replica Level 5. At each replica, enable requests for mastership at the replica level. multitool reqmaster –enable vob-selector The vob-selector specifies a VOB family, and your current replica is enabled for mastership requests. You must enter this command on the VOB server host. To enable or disable permission at the replica level, you must be the VOB owner, root (UNIX), or a member of the ClearCase administrators group (Windows). Also, the replica must master its own replica object. In an administrative VOB scenario, you enable requests for mastership in the client VOB replicas. You do not have to enable requests in the administrative VOB replica unless it contains elements that are developed serially. 148 Administrator’s Guide: Rational ClearCase MultiSite After you enable requests for mastership, inform the appropriate developers about mastership requests and how and when to use them. Working On a Team in the Working in Base ClearCase part of Developing Software describes the procedures developers must use to request mastership. NOTE: The reqmaster command is a cleartool subcommand as well as a multitool subcommand, so developers who will request mastership do not have to install MultiSite software on their client hosts. On Windows, developers can request mastership from the Find Checkouts window, the Merge Manager, and the Version Tree Browser. 9.5 Customizing Synchronization Updates for Mastership Requests After a mastership request is processed at the master replica, sync_export_list is invoked to export an update packet to the replica at the requester’s site. You can customize the export by specifying one or more of the options and arguments that are valid for sync_export_list, except for –replicas, which is always the replica at the requester’s site. To specify options and arguments for the export: 1. On the VOB server host of the exporting replica, edit the file /var/adm/atria/config/rfm_shipping.conf (UNIX) or ccase-home-dir\var\config\rfm_shipping.conf (Windows). 2. Add the options and arguments to the following line: RFM_OPTIONAL_ARGUMENTS = For example, to compress update packets: RFM_OPTIONAL_ARGUMENTS = -compress To suppress informational messages, use a specific shipping class (in this example, reqmaster) and compress update packets: RFM_OPTIONAL_ARGUMENTS = -quiet 1 -compress -sclass reqmaster On UNIX, MultiSite installation creates the file ccase-home-dir/config/services/rfm_shipping.template. If /var/adm/atria/config/rfm_shipping.conf does not exist, the installation creates it by copying the template file. If /var/adm/atria/config/rfm_shipping.conf exists, a note is printed in the installation log advising you to compare the existing file to the template and make any necessary changes. 9 - Implementing Requests for Mastership 149 On Windows, MultiSite installation creates the file ccase-home-dir\config\services\rfm_shipping.template. If ccase-home-dir\var\config\rfm_shipping.conf does not exist, the installation creates it by copying the template file. If ccase-home-dir\var\config\rfm_shipping.conf exists, you must compare the existing file to the template and make any necessary changes. 9.6 Displaying Mastership Request Settings To display the mastership request setting for a replica, branch type, or branch, use the describe command or the Mastership tab in the Properties Browser (Windows). These settings are also displayed in the Request Mastership dialog box on Windows. By default, the output from describe shows the mastership request setting. You can also use the –fmt option and specify %[reqmaster]p to display only the mastership request setting. For example: ➤ To display a replica’s mastership request setting: cleartool describe replica:boston_hub@/vobs/doc replica "boston_hub" created 15-Aug-00.14:19:03 by Susan Goechs (susan.user@minuteman) replica type: unfiltered master replica: boston_hub@/vobs/doc request for mastership: enabled owner: susan group: user host: "minuteman" identities: preserved feature level: 2 connectivity: connected Attributes: FeatureLevel = 2 cleartool describe –fmt "%[reqmaster]p\n" replica:sanfran_hub@/vobs/dev disabled 150 Administrator’s Guide: Rational ClearCase MultiSite ➤ To display a branch type’s mastership request setting: cleartool describe brtype:main@/vobs/doc branch type "main" created 15-Aug-00.14:19:03 by Susan Goechs (susan.user@minuteman) "Predefined branch type used to represent the main branch of elements." master replica: boston_hub@/vobs/doc request for mastership: allowed for branch type request for mastership: allowed for all instances ... cleartool describe –fmt "%[reqmaster]p\n" brtype:boston_main@/vobs/dev denied for all instances ➤ To display a branch’s mastership request setting: cleartool describe /vobs/doc/admin/setup.doc@@/main branch "/vobs/doc/admin/setup.doc@@/main" ... request for mastership: allowed ... cleartool describe –fmt "%[reqmaster]p\n" /vobs/doc/planning/plans.doc@@/main denied 9.7 Troubleshooting This section describes commands you can use to troubleshoot failed mastership requests, and lists error messages and their meanings. Troubleshooting Commands To determine which replica masters a branch or branch type: ➤ Use the cleartool describe command. For example: cleartool describe –fmt "%[master]p\n" file1.txt@@\main boston_hub@\dev 9 - Implementing Requests for Mastership 151 cleartool describe –fmt "%[master]p\n" brtype:main@/vobs/doc boston_hub@/vobs/doc ➤ (Windows) Display properties of the branch or branch type and click the Mastership tab. To determine whether a mastership request will succeed: ➤ Use reqmaster –list (see Status Messages on page 152 for descriptions of the output): multitool reqmaster –nc –list file1.txt@@/main multitool: Error: The following errors will be encountered multitool: Error: file1.txt@@/main Request Mastership remote "reqmaster" operation (host "taronga") would fail: You do not have permission to request mastership from the sibling replica. ➤ (Windows) In the Request Mastership dialog box, click Preview Request for Mastership. To list the event history of a branch or branch type and determine who has requested its mastership, use the lshistory –minor –fmt command: cleartool lshistory –min –fmt "%n\t%o\n%c" file.fm@@/main file.fm@@/main chmaster Reqmaster changed master replica from "boston_hub" to "buenosaires". Requester: user "PURPLEDOC\fangio" in domain "PURPLEDOC" on host "mardelplata" file.fm@@/main chmaster Reqmaster changed master replica from "tokyo" to "boston_hub". Requester: user "PURPLEDOC\susan" in domain "PURPLEDOC" on host "minuteman" file.fm@@/main chmaster Reqmaster changed master replica from "bangalore" to "tokyo". Requester: user "PURPLEDOC\masako" in domain "PURPLEDOC" on host "shinjuku" file.fm@@/main chmaster Reqmaster changed master replica from "sanfran_hub" to "bangalore". Requester: user "PURPLEDOC\kumar" in domain "PURPLEDOC" on host "ramohalli" ... cleartool lshistory –min –fmt "%n\t%o\n%c" brtype:main@/vobs/doc Status Messages Table 12 describes error messages you may see when you enable or disable requests at the replica level, work with the ACL, and allow or deny requests at the branch type or branch level. Table 13 describes error messages associated with mastership requests. 152 Administrator’s Guide: Rational ClearCase MultiSite Errors that occur during the mastership request process, including errors that occur during the synchronization export, are written to the msadm log file. To view it, use the cleartool getlog command or the ClearCase Administration Console (Windows). Table 12 Error Messages from Mastership Request Management Operations (Part 1 of 2) Message Meaning of message and action to take Could not check Request for Mastership permissions. The process that checks the ACL could not determine whether you have read or write permissions on the ACL. Check the msadm and albd log files on the client and server hosts and try the command again later. Could not edit Request Mastership ACL. You do not have permission to edit the ACL. To edit the ACL, you must be VOB owner, root (UNIX), a member of the ClearCase administrators group (Windows), or have write permission on the ACL. Could not get Request Mastership ACL. Your client computer could not retrieve the ACL from the VOB server host. There may be a network connection problem. Check the msadm and albd log files on the client and server hosts and try the command again later. Could not resolve object 'object-identifier'. The command could not find the object. Check the spelling of the object selector. In a dynamic view context, mount the VOB and try the command again. Object must be a branch or branch type. Specify a branch or branch type. Examples of branch specifications: /vobs/dev/acc.c@@/main (UNIX) \doc\stage.pl@@\main\debug (Windows) Examples of branch type specifications: brtype:main brtype:boston_main@/vobs/dev (UNIX) brtype:v1.0_bugfix@\tests (Windows) Request for mastership ACL operations on multiple replicas are not allowed. Specify only one VOB selector. The specified selector must be a VOB selector. Specify a VOB selector. For example: vob:/vobs/dev (UNIX) vob:\tests (Windows) Request for mastership ACL operations require a VOB-selector argument. 9 - Implementing Requests for Mastership 153 Table 12 Error Messages from Mastership Request Management Operations (Part 2 of 2) Message Meaning of message and action to take The VOB family feature level is too low to enable requests for mastership. The VOB family feature level is less than 2. If all replicas in the VOB family are at feature level 2 or greater, raise the family feature level. If any replica in the VOB family has a feature level less than 2, ask the administrator of that replica to upgrade to a newer version of Rational ClearCase (if necessary), raise the feature level of the replica, and send an update packet to the sibling replicas. Raise the family feature level. This replica does not master its replica object. A replica must be self-mastering for you to enable requests for mastership in that replica. See Transferring Mastership of a Replica Object on page 128. This replica does not master the branch. For you to allow or deny mastership requests for a branch, your current replica must master the branch. Determine which replica masters the branch and ask the administrator of the replica to change mastership of the branch to your replica. This replica does not master the branch type. For you to allow or deny mastership requests for a branch type, your current replica must master the branch type. Determine which replica masters the branch type and ask the administrator of the replica to change mastership of the branch type to your replica. You cannot specify -instances with the -enable option. To enable requests at the replica level, use the –enable option and specify a VOB selector. To deny or allow requests for all instances of a branch type, use the –deny or –allow option with the –instances option and specify a branch type selector. 154 Administrator’s Guide: Rational ClearCase MultiSite Table 13 Error Messages from Mastership Requests (Part 1 of 2) Message Meaning of message and action to take An error at the sibling replica prevented the request for mastership. The error cannot be specified. Try the request again later. If the request continues to fail, ask the administrator of the master replica to check the ClearCase and MultiSite log files. At least one checkout prevents the request. There is a blocking checkout on the branch being requested or on an instance of the branch type being requested. Try the request again later. If the request continues to fail, ask the user at the sibling replica to check in the element. Could not resolve object 'object-identifier'. The command could not find the object. Check the spelling of the object selector. Locks at the sibling replica prevented the request for mastership. A request for mastership fails if the branch or branch type is locked at the master replica. Ask the administrator of the master replica to unlock the branch or branch type. Requests are denied for all objects mastered by the sibling replica. Mastership requests are not enabled for the replica on host hostname. Ask the administrator of the master replica of the branch or branch type to enable mastership requests at the replica level. Requests are denied for all objects of the given type. Mastership requests are denied for all instances of the branch type. Ask the administrator of the master replica of the branch to use reqmaster –allow –instances or the Properties Browser (Windows) to allow requests for all instances. Requests are denied for the object. Mastership requests are denied for the branch or branch type. Ask the administrator of the master replica to use reqmaster –allow or the Properties Browser (Windows) to allow requests for the branch or branch type. Requests for mastership can be made only for branches and branch types. The user must specify a branch or branch type in the reqmaster command. Examples of branch specifications: /vobs/dev/acc.c@@/main (UNIX) \doc\stage.pl@@\main\debug (Windows) Examples of branch type specifications: brtype:main brtype:boston_main@/vobs/dev (UNIX) brtype:v1.0_bugfix@\tests (Windows) The object is not a branch or a branch type. 9 - Implementing Requests for Mastership 155 Table 13 Error Messages from Mastership Requests (Part 2 of 2) Message Meaning of message and action to take The object is already mastered by replica 'replica-selector'. The user’s current replica already masters the requested object. The object was not found at the sibling replica. This may indicate that the replicas are not in sync. The user’s current replica has more up-to-date information than other replicas in the VOB family. Ask the administrator of the current replica to do both of the following things: The sibling replica does not master the object. You do not have permission to request mastership from the sibling replica. 156 ➤ Verify that no update packets are waiting to be imported at other replicas in the VOB family. ➤ Determine whether update packets must be sent more frequently. (Frequent exchange of packets means that replicas have up-to-date information about the state of other replicas.) The user’s current replica has out-of-date information about the mastership of the object. Ask the administrator of the current replica to do both of the following things: ➤ Verify that no update packets are waiting to be imported at your current replica or the sibling replica. ➤ Send update packets more frequently. (Frequent exchange of packets means that replicas have up-to-date information about the state of other replicas.) The user requesting mastership is not included on the replica’s access control list. Ask the administrator of the sibling replica to use reqmaster –acl –get to display the access control list and check the following things: ➤ Spelling of user and domain names ➤ All variants of the domain name are included ➤ User’s access level Administrator’s Guide: Rational ClearCase MultiSite 9.8 Serial Development Scenario This section describes an example of serial development using requests for mastership. Planning the Implementation The company PurpleDoc develops documentation at three sites. There are two VOB families: ➤ /vobs/doc contains binary files. This VOB has three replicas: boston_hub, tokyo, and sanfran_hub. The writers working in /vobs/doc use serial development because the files are in binary format. However, a team of writers at the Boston site needs control of a certain set of files at all times. ➤ /vobs/html contains html files and scripts. This VOB has three replicas: boston_hub, tokyo, and sanfran_hub. The writers working on HTML files in /vobs/html use site-specific branch types: boston_main, tokyo_main, and sanfran_main. Writers at a particular site cannot use branch types mastered by the other sites. The tool developers working on scripts use the main branch. Because the scripts can be merged, the developers can use nonmastered checkouts to do their work. Setting Up Access Controls The administrators and project managers at the Boston, San Francisco, and Tokyo sites make the following decisions: ➤ Writers are allowed to request mastership of all branches in /vobs/doc, except for the branches plans.doc@@/main, schedule.doc@@/main, and roadmap.doc@@/main. ➤ Writers are not allowed to request mastership of any branches of type boston_main, tokyo_main, or sanfran_main in /vobs/html. ➤ Tool developers are allowed to request mastership of all branches of type main in /vobs/html. 9 - Implementing Requests for Mastership 157 Each administrator completes the following steps on the replica’s VOB server host. (This example takes place at the Boston site.) 1. Add writers at other sites to the ACL for /vobs/doc. a. Place the following lines in the file /tmp/doc_acl: # Replica boston_hub@/vobs/doc # Request for Mastership ACL: User:boston.purpledoc.com/msadm Full User:tokyo.purpledoc.com/masako Change User:tokyo.purpledoc.com/sato Change User:tokyo.purpledoc.com/ito Change User:sf.purpledoc/jcole Change User:sf.purpledoc/marni Change User:sf.purpledoc/david Change b. Use the file to set the replica’s ACL: multitool reqmaster –acl –set /tmp/doc_acl vob:/vobs/doc 2. Add tool developers at other sites to the ACL for /vobs/html. a. Place the following lines in the file /tmp/html_acl: # Replica boston_hub@/vobs/html # Request for Mastership ACL: User:boston.purpledoc.com/ccadmin Full User:tokyo.purpledoc.com/masako Change User:sf.purpledoc/david Change b. Use the file to set the replica’s ACL: multitool reqmaster –acl –set /tmp/html_acl vob:/vobs/html NOTE: After you set the ACL, you can delete the temporary ACL files you created. 3. Deny mastership requests for specific branches and branch types: multitool reqmaster –deny /vobs/doc/planning/plans.doc@@/main \ /vobs/doc/planning/schedule.doc@@/main /vobs/doc/planning/roadmap.doc@@/main multitool reqmaster –deny –instances brtype:boston_main@/vobs/html multitool reqmaster –deny brtype:boston_main@/vobs/html 158 Administrator’s Guide: Rational ClearCase MultiSite 4. Enable requests for mastership at the replica level. multitool reqmaster –enable vob:/vobs/doc vob:/vobs/html Writing Config Specs In this scenario, the writers use the config specs listed below. Each location has rules for creating site-specific branches in /vobs/html and selecting the latest version on that branch. The /main/LATEST rule is used in all the config specs for development in /vobs/doc and all other VOBs. Boston element element element element element * CHECKEDOUT /vobs/html/scripts/... /main/LATEST /vobs/html/files/... /main/boston_main/LATEST /vobs/html/files/... /main/LATEST -mkbranch boston_main * /main/LATEST San Francisco element element element element element * CHECKEDOUT /vobs/html/scripts/... /main/LATEST /vobs/html/files/... /main/sanfran_main/LATEST /vobs/html/files/... /main/LATEST -mkbranch sanfran_main * /main/LATEST Tokyo element element element element element * CHECKEDOUT /vobs/html/scripts/... /main/LATEST /vobs/html/files/... /main/tokyo_main/LATEST /vobs/html/files/... /main/LATEST -mkbranch tokyo_main * /main/LATEST 9 - Implementing Requests for Mastership 159 Requesting Mastership The following sections describe how the writers use mastership requests to do their work. Serial Development of a File That Cannot Be Merged 1. Masako, in Tokyo, tries to check out the file \doc\ref\update.fm, but the checkout fails because the Tokyo replica doesn’t master the main branch: cleartool checkout –c "new command options" \doc\ref\update.fm cleartool: "tokyo" of cleartool: cleartool: Error: Unable to perform operation "checkout" in replica VOB "\doc". Error: Master replica of branch "\main" is "boston_hub". Error: Unable to check out "\doc\ref\update.fm". 2. She requests mastership of branch \doc\ref\update.fm@@\main: cleartool reqmaster –c "Tokyo needs mastership" \doc\ref\update.fm@@\main 3. Periodically, she retries the checkout or displays properties of the branch to determine whether mastership has been received. cleartool checkout –c "new command options" \doc\ref\update.fm cleartool: "tokyo" of cleartool: cleartool: Error: Unable to perform operation "checkout" in replica VOB "\doc". Error: Master replica of branch "\main" is "boston_hub". Error: Unable to check out "\doc\ref\update.fm". After mastership is received at her replica, the describe command shows that her replica masters the branch and her checkout succeeds: cleartool describe –fmt "%[master]p\n" \doc\ref\update.fm@@\main tokyo@\doc cleartool checkout –c "new command options" \doc\ref\update.fm Checked out "\doc\ref\update.fm" from version "\main\30". 160 Administrator’s Guide: Rational ClearCase MultiSite Serial Development of a File That Can Be Merged 1. John, in San Francisco, needs to change an HTML script. He can’t check out the file using a reserved checkout because the branch is mastered by the Boston replica: cleartool checkout –c "option to suppress status msgs" /vobs/html/scripts/conv_fm.pl cleartool: Error: Unable to perform operation "checkout" in replica "sanfran_hub" of VOB "/vobs/html". cleartool: Error: Master replica of branch "/main" is "boston_hub". cleartool: Error: Unable to check out "/vobs/html/scripts/conv_fm.pl". 2. He requests mastership of the branch: cleartool reqmaster –c "SF: add new option" /vobs/html/scripts/conv_fm.pl@@/main 3. He checks out the file with the –unreserved and –nmaster options and proceeds to edit the file: cleartool checkout –c "option to suppress status msgs" –unreserved \ –nmaster /vobs/html/scripts/conv_fm.pl Checked out "/vobs/html/scripts/conf_fm.pl" from version "/main/15". 4. Until mastership is received at the San Francisco replica, he cannot check in his changes: cleartool checkin –nc conv_fm.pl cleartool: Error: Unable to perform operation "checkin" in replica "sanfran_hub" of VOB "/vobs/html". cleartool: Error: Master replica of branch "/main" is "boston_hub". cleartool: Error: Unable to check in "conv_fm.pl". 5. When mastership is received at the San Francisco replica, he attempts to check in the file, but finds that he must perform a merge: cleartool checkin –nc conv_fm.pl cleartool: Error: The most recent version on branch "/main" is not the predecessor of this version. cleartool: Error: Unable to check in "conv_fm.pl". 9 - Implementing Requests for Mastership 161 6. He performs the merge, and checks in the file: cleartool merge –to conv_fm.pl –c "merging from LATEST" –version /main/LATEST ******************************** <<< file 1: /vobs/html/conv_fm.pl@@/main/15 >>> file 2: /vobs/html/conv_fm.pl@@/main/16 >>> file 3: conv_fm.pl ******************************** . . . Moved contributor "conv_fm.pl" to "conv_fm.pl.contrib". Output of merge is in "conv_fm.pl". Recorded merge of "conv_fm.pl". cleartool checkin –nc conv_fm.pl Checked in "conv_fm.pl" version "/main/17". 162 Administrator’s Guide: Rational ClearCase MultiSite Troubleshooting MultiSite Operations 10 10 This chapter describes common situations in which running a MultiSite command produces an unexpected result, sometimes accompanied by a warning or error message. The situations fall into these categories: ➤ Expected conditions occur because certain inconsistent changes at different replicas cannot be avoided. In many cases, a MultiSite operation resolves the inconsistency, so you need not take any action. ➤ Recoverable errors are user errors, hardware glitches, and other problems that you resolve by performing a recovery procedure. ➤ Serious errors are problems that may require assistance from Rational Technical Support. The organization of the descriptions follows the general MultiSite data flow: from replica creation through the phases of replica synchronization—export, transport, and import. This chapter also describes replica restoration and replacement. For information about changing mastership, see Chapter 8, Managing Mastership. For information about mastership request errors, see Chapter 9, Implementing Requests for Mastership. 10 - Troubleshooting MultiSite Operations 163 10.1 Troubleshooting Tips Use the following files and commands to help diagnose MultiSite problems: ➤ Log files. To view log files, use the cleartool getlog command or the ClearCase Administration Console (Windows). ➣ MultiSite log files Export/import problems Transport problems Mastership request problems Other errors ➣ Files in directory /var/adm/atria/log/sync_logs (UNIX) or ccase-home-dir\var\log (Windows) shipping msadm Command window Event Viewer (Windows) ClearCase log files. If ClearCase problems affect MultiSite operation (for example, a MultiSite operation fails when the ClearCase db_server cannot process the VOB database), useful information appears in these log files. ➤ Make sure you install the latest ClearCase and MultiSite patches. ➤ Most MultiSite commands do not require a view context or a mounted VOB replica. If a command such as syncreplica –import fails, you can produce better diagnostics by following the steps below. On UNIX: a. Set a dynamic view or change to a directory within a snapshot view. b. Mount the VOB replica (dynamic view) or load a single file in the VOB (snapshot view). c. Change into a directory in the replica. If you used a snapshot view, this must be the directory containing the file you loaded. d. Enter the command again. On Windows: a. Change to a directory within a snapshot view or to a view drive. b. Mount the VOB replica (dynamic view) or load a single file in the VOB (snapshot view). 164 Administrator’s Guide: Rational ClearCase MultiSite c. Change into a directory below the root directory. If you used a snapshot view, this must be the directory containing the file you loaded. d. Enter the command again. ➤ The commands listed below provide valuable information, especially if you are sending data to Rational Technical Support: multitool –version multitool lsreplica multitool lsepoch uname –a (UNIX) cleartool –version On Windows, look for applicable messages in the Event Viewer’s application log and system log, and in the ClearCase MVFS log files (c:\mvfslogs). 10.2 Replica-Creation Problems Problems with replica creation can occur during the export phase or the import phase. Export Phase If the mkreplica –export command finds that a replica with the specified name exists in the VOB family (Replica replica-name already exists), select another name for the new replica, and reenter the mkreplica –export command. If mkreplica –export –fship fails while it is transporting the packet, it does not remove the new replica’s replica object at the creating site. To complete the replica creation, use shipping_server to transfer the replica-creation packet. 10 - Troubleshooting MultiSite Operations 165 Import Phase A recoverable error occurs if the mkreplica –import command detects a conflict at the ClearCase registry level—an entry exists in the VOB object registry or in the tags registry: Replica replica-name already exists Conflict in VOB Object Registry A conflict in the registry can occur if a mkreplica –import commands fails and removes the VOB storage directory but not the registry entry. Verify that cleartool lsvob does not report any VOB storage directory at the location you specified with the –vob option. In this case, the VOB object registry contains an entry with no corresponding VOB-tag. For example: cleartool lsvob –storage /net/goldengate/vobstg/dev.vbs cleartool: Error: Unable to access "/net/goldengate/vobstg/dev.vbs": No such file or directory. cleartool: Error: Versioned object base not found: "/net/goldengate/vobstg/dev.vbs". cleartool: Error: No vob tags found for vob "/net/goldengate/vobstg/dev.vbs". Restore the registry to a consistent state by following these steps: 1. In the VOB object registry file, find the incorrect entry for the VOB storage directory pathname you specified. This file is located on the network’s registry server host in /var/adm/atria/rgy/vob_object on UNIX or ccase-home-dir\var\rgy\vob_object on Windows. 2. Using the UUID in this entry, enter a cleartool unregister –vob –uuid command to remove the incorrect entry. CAUTION: Do not edit the information in the registry file directly. 3. With the registry restored to a consistent state, reenter the mkreplica –import command. 4. After the mkreplica command succeeds, delete the replica-creation packet from disk storage (if appropriate). 166 Administrator’s Guide: Rational ClearCase MultiSite Conflict in VOB-Tag Registry mkreplica –import may be able to create and register the VOB storage directory, but may find that the specified VOB-tag is already in use. In this case, create another VOB-tag for the new VOB storage directory with a cleartool mktag command or with the ClearCase Administration Console (available on Windows). You do not have to reenter the mkreplica –import command in this case. You can delete the replica-creation packet from disk storage (if appropriate). 10.3 Synchronization Export Problems This section describes problems that can occur during the export phase of synchronization. To list the exports from your current replica to a sibling replica, use the following command: cleartool lshistory replica:sibling-replica-name@vob-selector For example, to list exports from your current replica in VOB family /vobs/dev to the replica sanfran_hub: cleartool lshistory replica:sanfran_hub@/vobs/dev 12-Jul.16:13 root export sync from replica "boston_hub" to replica "sanfran_hub" "Exported synchronization information for replica "sanfran_hub". Row at export was: boston_hub=149 sanfran_hub=115" 29-Jun.16:19 smg change epoch of replica "sanfran_hub" "Changed epoch row for replica Old row was: boston_hub=152 sanfran_hub=115 New row is: boston_hub=149 sanfran_hub=115 epoch row set by special connected epoch tool." 29-Jun.10:12 smg export sync from replica "boston_hub" to replica "sanfran_hub" "Exported synchronization information for replica "sanfran_hub". Row at export was: boston_hub=149 sanfran_hub=115" ... 10 - Troubleshooting MultiSite Operations 167 Cannot Find Oplog syncreplica –export can fail with the following warning message: Can not find oplog from replica replica-name with id oplog-ID Gap in oplog entries may indicate missing oplog entries (For more information on oplog entries, see VOB Operations and the Oplog on page 24 and Scrubbing Parameters for VOB Replicas on page 47.) This error occurs when the sending replica’s epoch number matrix does not match its set of oplog entries. For example: ➤ Before sending an update from sydney to buenosaires, syncreplica checks the epoch number matrix for sydney. It determines that the last sydney operation sent to buenosaires was 3620. ➤ syncreplica finds that oplog scrubbing in the sydney database has removed some of the operations that follow 3620. The earliest sydney operation remaining in the oplog is 5755. This discrepancy may be an expected condition. For example, when a VOB family changes its update topology, hosts that have not communicated with each other in the past start exchanging update packets. Synchronizing two replicas (syncreplica –export followed by syncreplica –import) updates epoch number matrix rows for the sending and receiving replicas, but it does not revise the row for any other replica. If two replicas rarely (or never) send updates to each other directly, the relevant rows in their epoch number matrices are out of date (possibly consisting of all zeros). This is not a problem, as long as the replicas receive operations indirectly, for example, through a hub replica. In this case, you must inform sydney about the true state of buenosaires, information that it has not received through the standard synchronization-update mechanism. This information enables sydney to determine which oplog entries must be sent to buenosaires. If the sites have an IP connection, use the procedure in Sites Have IP Connection. If the sites do not have an IP connection, use the procedure in Sites Do Not Have IP Connection. Sites Have IP Connection At sydney, use the chepoch –actual command to contact buenosaires, retrieve its actual state, and reset the epoch row for buenosaires. multitool chepoch –actual replica:buenosaires@/vobs/tests 168 Administrator’s Guide: Rational ClearCase MultiSite Sites Do Not Have IP Connection Proceed as follows: 1. At buenosaires, use the lshistory command to determine when the last update packet was processed successfully. cleartool lshistory replica:buenosaires@/vobs/tests 01-Sep.01:00 garyf import sync from replica “sydney” to replica “buenosaires” “Imported synchronization information from replica “sydney”. Row at import was: buenosaires=8 sydney=3 boston_hub=0” 01-Aug.07:05 garyf import sync from replica “boston_hub” to replica “buenosaires” “Imported synchronization information from replica “boston_hub”. Row at import was: buenosaires=2 boston_hub=0” 01-Jul.15:55 garyf create replica “buenosaires” 2. At sydney, use this time in a recoverpacket command to reset the epoch row for buenosaires. Assume that the sydney site is thirteen hours ahead of the buenosaires site. multitool recoverpacket –since 01-Sep.14:00 buenosaires If this command succeeds, proceed to Step #3. If this command fails: a. At buenosaires (destination site), run lsepoch to determine the actual state of buenosaires: multitool lsepoch buenosaires@/vobs/tests b. Send the lsepoch command output back to the sending site, where the administrator of sydney uses this data in a chepoch command to inform sydney about the actual state of buenosaires. cd /vobs/dev multitool chepoch buenosaires Enter specifications for epochs to change in row "buenosaires" (one per line)
Source Exif Data:
File Type : PDF File Type Extension : pdf MIME Type : application/pdf PDF Version : 1.2 Linearized : No Page Count : 348 Page Mode : UseOutlines Producer : Acrobat Distiller Command 3.01 for Solaris 2.3 and later (SPARC) Creator : FrameMaker 5.5.6. Modify Date : 2001:09:10 17:48:20 Create Date : 2001:08:30 18:16:04EXIF Metadata provided by EXIF.tools