BDB Porting Guide
BDB-Porting-Guide
User Manual:
Open the PDF directly: View PDF .
Page Count: 16
Download | |
Open PDF In Browser | View PDF |
Oracle Berkeley DB Porting Guide 12c Release 1 Library Version 12.1.6.2 Legal Notice This documentation is distributed under an open source license. You may review the terms of this license at: http:// www.oracle.com/technetwork/database/berkeleydb/downloads/oslicense-093458.html Oracle, Berkeley DB, and Sleepycat are trademarks or registered trademarks of Oracle. All rights to these marks are reserved. No third-party use is permitted without the express prior written consent of Oracle. Other names may be trademarks of their respective owners. To obtain a copy of this document's original source code, please submit a request to the Oracle Technology Network forum at: https://forums.oracle.com/forums/forum.jspa?forumID=271 Published 3/28/2016 Table of Contents Preface ...................................................................................................... iv Conventions Used in this Book .................................................................... iv Audience .............................................................................................. iv For More Information ............................................................................... v Contact Us ...................................................................................... v 1. Introduction to Porting Berkeley DB ................................................................ 1 Types of Berkeley DB ports ........................................................................ 1 When Oracle Has Agreed to Support Berkeley DB on the New Platform .............. 1 When Oracle has Not Agreed to Support Berkeley DB on the New Platform ......... 2 Berkeley DB Porting Process ....................................................................... 2 2. Creating a New Berkeley DB Binary ................................................................. 3 Creating a Base Build of Berkeley DB .......................................................... 3 Determining the Scope of the Modifications .................................................. 3 Do Changes Need to be Made to the Operating System Functionality? .............. 4 Are Some Standard Functions Missing on the Target Platform? ........................ 6 How Will the Port Handle Shared Memory? ............................................... 6 What Type of Mutexes Will the Port Use? ................................................. 6 Do Any Other Changes Need to be Made? ................................................. 7 Building on the Target Platform ................................................................. 7 Source Code Layout ................................................................................. 7 3. Testing and Certifying the Port ...................................................................... 8 Types of Tests for Berkeley DB .................................................................... 8 Modifying the Tests .................................................................................. 8 Running the Tests .................................................................................... 8 Reviewing the Results of the Tests ............................................................... 9 Integrating Changes into the Berkeley DB Source Code ....................................... 9 Certifying a Port of Berkeley DB ................................................................ 10 3/28/2016 DB Porting Guide Page iii Preface The Berkeley DB family of open source, embeddable databases provides developers with fast, reliable persistence with zero administration. Often deployed as "edge" databases, the Berkeley DB family provides very high performance, reliability, scalability, and availability for application use cases that do not require SQL. As an open source database, Berkeley DB works on many different platforms, from Wind River's VxWorks system, to Windows, to Android, and on most existing UNIX platforms. It runs on 32 and 64-bit machines, little or big-endian. Berkeley DB Porting Guide provides the information you need to port Berkeley DB 12c Release 1 (library version 12.1.6.2) to additional platforms. Conventions Used in this Book The following typographical conventions are used within in this manual: Structure names are represented in monospaced font, as are method names. For example: "DB->open() is a method on a DB handle." Variable or non-literal text is presented in italics. For example: "Go to your DB_INSTALL directory." Program examples are displayed in a monospaced font on a shaded background. For example: /* File: gettingstarted_common.h */ typedef struct stock_dbs { DB *inventory_dbp; /* Database containing inventory information */ DB *vendor_dbp; /* Database containing vendor information */ char *db_home_dir; /* Directory containing the database files */ char *inventory_db_name; /* Name of the inventory database */ char *vendor_db_name; /* Name of the vendor database */ } STOCK_DBS; Note Finally, notes of interest are represented using a note block such as this. Audience This guide is intended for programmers porting Berkeley DB to a new platform. It assumes that these programmers possess: • Familiarity with standard ANSI C and POSIX C 1003.1 and 1003.2 library and system calls. • Working knowledge of the target platform as well as the development tools (for example, compilers, linkers, and debuggers) available on that platform. 3/28/2016 DB Porting Guide Page iv For More Information Beyond this manual, you may also find the following sources of information useful when building a DB application: • Getting Started with Berkeley DB for C • Getting Started with Transaction Processing for C • Berkeley DB Getting Started with Replicated Applications for C • Berkeley DB Installation and Build Guide • Berkeley DB Programmer's Reference Guide • Berkeley DB Getting Started with the SQL APIs • Berkeley DB C API Reference Guide To download the latest Berkeley DB documentation along with white papers and other collateral, visit http://www.oracle.com/technetwork/indexes/documentation/index.html. For the latest version of the Oracle Berkeley DB downloads, visit http://www.oracle.com/ technetwork/database/database-technologies/berkeleydb/downloads/index.html. Contact Us You can post your comments and questions at the Oracle Technology (OTN) forum for Oracle Berkeley DB at: https://forums.oracle.com/forums/forum.jspa?forumID=271, or for Oracle Berkeley DB High Availability at: https://forums.oracle.com/forums/forum.jspa?forumID=272. For sales or support information, email to: berkeleydb-info_us@oracle.com You can subscribe to a low-volume email announcement list for the Berkeley DB product family by sending email to: bdb-join@oss.oracle.com 3/28/2016 DB Porting Guide Page v Chapter 1. Introduction to Porting Berkeley DB Berkeley DB is an open source database product that supports a variety of platforms. When there is a need to run Berkeley DB on a platform that is currently not supported, DB is distributed in source code form that you can use as base source to port Berkeley DB to that platform. Berkeley DB is designed to be as portable as possible, and has been ported to a wide variety of systems, from Wind River's Tornado system, to VMS, to Windows/NT and Windows/95, and most existing UNIX platforms. It runs on 16-bit, 32-bit, and 64-bit machines, little or big-endian. The difficulty of a port depends on how much of the ANSI C and POSIX 1003.1 standards the new architecture offers. Before you begin actually porting Berkeley DB, you need an understanding of the: • Types of Berkeley DB ports (page 1) • Berkeley DB Porting Process (page 2) Types of Berkeley DB ports There are several types of Berkeley DB ports: • Ports developed and supported by Oracle • Ports developed by a customer or a partner, but which Oracle has agreed to support. • Ports developed, maintained, and supported by a customer or partner. For a port developed by a customer or a partner, the general steps for porting Berkeley DB to a new platform are the same whether or not Oracle has agreed to support Berkeley DB on the new platform. For example, after you complete the port you send it to Berkeley DB as described in Integrating Changes into the Berkeley DB Source Code (page 9). However, there are some differences. When Oracle Has Agreed to Support Berkeley DB on the New Platform When porting Berkeley DB to a platform that Oracle has agreed to support, you need to have Berkeley DB engineering review your port at various points. These review points are discussed more fully in Integrating Changes into the Berkeley DB Source Code (page 9), Modifying the Tests (page 8), and Reviewing the Results of the Tests (page 9). It is up to you to submit the results of the tests (test_micro, test_mutex, and, if possible, the entire tcl test suite) for review by Oracle Berkeley DB engineering in order for Oracle to consider providing support for Berkeley DB on a new platform. You must also assign copyrights for your changes to any part of Berkeley DB to "Oracle Corporation" and attest to the fact that you are not infringing on any software patents for the changes to be included in the general Berkeley DB distribution. 3/28/2016 DB Porting Guide Page 1 Introduction to Porting Berkeley DB Library Version 12.1.6.2 Once the port is certified, Oracle provides support for Berkeley DB on the new platform in the same manner that it does for Berkeley DB running on other established platforms. When Oracle has Not Agreed to Support Berkeley DB on the New Platform When Oracle has not agreed to support Berkeley DB on the new platform, the customer or partner assume the responsibility of front-line support. When it is determined that there is a problem in the code that was not modified by the customer or partner, then Berkeley DB engineering provides support to the customer or vendor who implemented the port, However, in these cases, Oracle needs access to the platform and hardware for diagnosing, debugging, and testing. Berkeley DB Porting Process As with any porting project, porting Berkeley DB to a new platform consists of the following process: 3/28/2016 1. Determine the modifications that you need to make in the base code and create a working executable of Berkeley DB on the target platform as described in Creating a New Berkeley DB Binary (page 3). 2. Perform final test and quality assurance functions on the target platform as described in Testing and Certifying the Port (page 8). DB Porting Guide Page 2 Chapter 2. Creating a New Berkeley DB Binary Creating a new Berkeley DB executable on the target platform, involves: 1. Creating a Base Build of Berkeley DB (page 3) 2. Determining the Scope of the Modifications (page 3) 3. Building on the Target Platform (page 7) Creating a Base Build of Berkeley DB The simplest way to begin a port is to attempt to configure and build Berkeley DB on a UNIX or UNIX-like system. This gives you a list of the files that you needed to build Berkeley DB as well as the configuration files you can use as a starting point for building on your target port. To create a base build of Berkeley DB, following the instructions in the Berkeley DB Programmer's Reference Guide: 1. Download a Berkeley DB distribution from http://www.oracle.com/technetwork/ database/database-technologies/berkeleydb/downloads/index.html. 2. Build Berkeley DB. Berkeley DB uses the GNU autoconf tools for configuration on almost all of the platforms it supports. Specifically, the include file db_config.h configures the Berkeley DB build. The simplest way to begin a port is to configure and build Berkeley DB on a UNIX or UNIX-like system, and then take the Makefile and db_config.h file created by that configuration, and modify it by hand to reflect the needs of the new architecture. Unless you are already familiar with the GNU autoconf toolset, we do not recommend you take the time to integrate your changes back into the Berkeley DB autoconfiguration framework. Instead, send us context diffs of your changes and any new source files you created, and we can integrate the changes into our source tree. Determining the Scope of the Modifications Once you have a good build of Berkeley DB on a UNIX or UNIX-like system, look over the code to determine what type of code changes you need to make so that you can successfully build Berkeley DB on your target system. This process involves determining: • Do Changes Need to be Made to the Operating System Functionality? (page 4) • Are Some Standard Functions Missing on the Target Platform? (page 6) • How Will the Port Handle Shared Memory? (page 6) • What Type of Mutexes Will the Port Use? (page 6) • Do Any Other Changes Need to be Made? (page 7) 3/28/2016 DB Porting Guide Page 3 Creating a New Berkeley DB Binary Library Version 12.1.6.2 Do Changes Need to be Made to the Operating System Functionality? Berkeley DB uses about forty operating system primitives. The Berkeley DB distribution contains files which are wrappers around these operating system primitives that act as an abstraction layer to separate the main Berkeley DB code from operating system and architecture-specific components. You must port these files (or versions of these files) whenever you port Berkeley DB to a new platform. Within a Berkeley DB distribution, typically, there is only a single version of these files for all platforms that Berkeley DB supports. Those versions of the files live in the os directory of the distribution and follow the ANSI C and POSIX 1003.1 standards. Within each file, there is usually one, but sometimes several functions (for example, the os_alloc.c file contains functions such as malloc, realloc, strdup, free). The following table describes the files in the os directory of the Berkeley DB distribution along with the POSIX functions that must be ported. POSIX Functions Internal Function Name Source File abort() is required if diagnostic build is used or if snprintf is not provided by the platform __os_abort() os_abort.c freeaddrinfo() __os_getaddrinfo(), __os_freeaddrinfo() os_addrinfo.c malloc(), realloc(), strdup(), free(), memcpy(), memset(), strlen() __os_umalloc(), __os_urealloc(), __os_ufree(), __os_strdup(), __os_calloc(), __os_malloc(), __os_realloc(), __os_free(), __os_guard(), __ua_memcpy() os_alloc.c clock_gettime(), time(), gettimeofday() __os_gettime() os_clock.c sysconf() __os_cpu_count() os_cpu.c ctime(), ctime_r() __os_ctime() os_ctime.c opendir(), closdir(), readdir(), stat() __os_dirlist(), __os_dirfree() os_dir.c strncpy() __os_get_errno_ret_zero(), __os_get_errno(), __os_get_syserr(), __os_set_errno(), __os_strerror(), __os_posix_err() os_errno.c fcntl() is required for DB_REGISTER __os_fdlock() os_flock.c fsync(), fdatasync() __vx_fsync(), __os_fsync() os_fsync.c getaddrinfo(), htonl(), htons(), inet_addr(), and gethostbyname() are required for Replication Manager getenv() and strcpy() are required __os_getenv() when environment variables are used to configure the database 3/28/2016 DB Porting Guide os_getenv.c Page 4 Creating a New Berkeley DB Binary Library Version 12.1.6.2 POSIX Functions Internal Function Name Source File close(), open() __os_openhandle(), __os_closehandle() os_handle.c getpid() __os_id() os_pid.c __os_attach(), __os_detach(), __os_mapfile(), __os_unmapfile(), __os_map(), __shm_mode(), __no_system_mem() os_map.c mkdir(), chmod() __os_mkdir() os_mkdir.c fchmod() __os_open() os_open.c rename() __os_rename() os_rename.c getuid() is required when environment variables are used to configure the database __os_isroot() os_root.c read(), write(), pread(), pwrite() __os_io(), __os_read(), __os_write(), os_rw.c __os_physwrite() lseek() __os_seek() os_seek.c stat(), fstat() __os_exists(), __os_ioinfo() os_stat.c ftruncate() is required when using truncate __os_truncate() os_truncate.c unlink() __os_unlink() os_unlink.c yield(), sched_yield() __os_yield(), __os_sleep() os.yield.c pthread_self() is required for replication and failchk functionality shmget(), shmdt(), shmctl(), and shmat() are required when envrionment uses share memory for regions munmap() is required when envrionment uses memory mapped files for regions or read-only databases munlock() is required when environment is configured with DB_LOCKDOWN directio() is required when explicitly enabling DIRECTIO_ON When the operating system primitives on the target platform are identical or close to the POSIX semantics that Berkeley DB requires, then no code changes or minimal code changes to the files in the os directory are required. If the operating system primitives are quite different, then some code changes may be required to bridge the gap between the requirements of Berkeley DB and what the operating system provides. 3/28/2016 DB Porting Guide Page 5 Creating a New Berkeley DB Binary Library Version 12.1.6.2 Where different code is required, you write an entirely different version of the file and place it in an os_xxx directory where xxx represents a platform name. There are os_xxx subdirectories in the Berkeley DB distribution for several established non-POSIX platforms. For example, there is a os_vxworks directory that contains VxWorks versions of some of the files in the os directory, and Windows versions of some files are in the os_windows directory. If your target platform needs a different version of a file, you will need to write that file and place it in a new os_xxx directory that you create for your target platform. Are Some Standard Functions Missing on the Target Platform? In some cases, the target platform may not provide the few POSIX functions required by Berkeley DB or the functions provided by the target platform may not operate in a standard compliant way. Berkeley DB provides replacement functions in the clib directory of the Berkeley DB distribution. You need to determine how your target platfrom handles these functions: • When the target platform does not have a POSIX function required by Berkeley DB, no action is required on your part. When Berkeley DB cannot find one of these functions on the target platform, it automatically uses the replacement functions supplied in the clib directory of the Berkeley DB distribution. For example, if the target platform does not have the atoi or strtol functions, Berkeley DB uses clib/atoi.c and clib/strtol.c. • When the target platform has a function required by Berkeley DB, but that function operates in a non-standard compliant way, you can code to the replacement functions supplied in the clib directory. How Will the Port Handle Shared Memory? In order to write multiprocess database applications (not multithreaded, but threads of control running in different address spaces), Berkeley DB must be able to name pieces of shared memory and access them from multiple processes. On UNIX/POSIX systems, Berkeley DB uses mmap and shmget for that purpose, but any interface that provides access to named shared memory is sufficient. If you have a simple, flat address space, you should be able to use the code in os_vxworks/os_map.c as a starting point for the port. If you are not intending to write multiprocess database applications, then this won't be necessary, as Berkeley DB can simply allocate memory from the heap if all threads of control will live in a single address space. What Type of Mutexes Will the Port Use? Berkeley DB requires some form of self-blocking mutual exclusion mutex. Blocking mutexes are preferred as they tend to be less CPU-expensive and less likely to cause thrashing. If blocking mutexes are not available, however, test-and-set will work as well. The code for mutexes is in two places in the system: the include file dbinc/mutex_int.h, and the distribution directory mutex. 3/28/2016 DB Porting Guide Page 6 Creating a New Berkeley DB Binary Library Version 12.1.6.2 Do Any Other Changes Need to be Made? In most cases, you do not need to make any changes to the Berkeley DB source code that is not in the abstraction layer (that is, in the os directory) as that code is designed to be platform-independent code. However, in some situations, the compiler for the target platform is non-standard and may raise errors when compiling some aspects of the Berkeley DB code (for example, additional casting may be required, or a certain type may cause a problem). In these cases, you will need to modify the generic Berkeley DB code in order to have error-free compilation. Building on the Target Platform Once you have an idea of the scope of the modifications, use the files generated on the UNIX system to help you create any compiler or linker tool chain files that you need to build on the target platform. At this point, start building the Berkeley DB on the target platform making the changes to the code that you identified earlier in the process. Once you have identified the modifications that you need to make, change the code accordingly. Source Code Layout The following table describes the directories in the Berkeley DB distribution. 3/28/2016 Directory Description LICENSE Berkeley DB License build_android Android build directory build_unix UNIX build directory build_vxworks VxWorks build directory build_wince Windows CE build directory build_windows Windows build directory dist Scripts used to auto-generate code for Berkeley DB distribution and administration docs Documentation examples Example programs for various language APIs lang Implementation of various APIs that work with the Berkeley DB library src Implementation of the Berkeley DB library test Test suites util Implementation of utilities that can be used with Berkeley DB DB Porting Guide Page 7 Chapter 3. Testing and Certifying the Port There are several different types of tests available for validating your port of Berkeley DB as discussed in Types of Tests for Berkeley DB (page 8). Testing your port involves: • Modifying the Tests (page 8) • Running the Tests (page 8) • Reviewing the Results of the Tests (page 9) • Integrating Changes into the Berkeley DB Source Code (page 9) • Certifying a Port of Berkeley DB (page 10) Types of Tests for Berkeley DB There are two types of tests available for testing your port of Berkeley DB: • The C Tests for Berkeley DB There are two types of C tests for Berkeley DB. Each of these is in its own directory: • The test_mutex test is found in src/mutex. • The test_micro tests are found in test/micro. Here you will find C tests that exercise the most common code paths. However, please note that test_micro is not intended to be an exhaustive test suite. Additionally, it tests the different versions of Berkeley DB (including the new port) against each other. The test_micro tests can either be run in a shell or as simple C tests. • The Berkeley DB Test Suite The test directory contains the Berkeley DB Test Suite that tests all of the code in Berkeley DB. Using the Test Suite involves using Tool Command Language (Tcl) version 8.5 or later. Running the standard version of the Test Suite executes tests the major functionality of Berkeley DB. A more exhaustive version of the Test Suite runs all the tests several more times, testing encryption, replication, and different page sizes. Modifying the Tests There should be no need to make modifications to the tests. However, in a few situations, the test hardware may have some constraints (for example, small amount of memory) which may cause certain tests (which expect more memory) to fail. In these rare situations, you may need to modify the tests to work within the constraints of the platform. When Oracle has agreed to support Berkeley DB on the new platform, submit any proposed test changes for review and approval by Oracle Engineering before running the tests. Running the Tests You test your new port of Berkeley DB by running the tests in the following order: 3/28/2016 DB Porting Guide Page 8 Testing and Certifying the Port Library Version 12.1.6.2 1. Run the C tests in the following order: a. Tests for mutexes located in the src/mutex directory. Build and run the tests in your build directory: % make test_mutex % ./test_mutex b. 2. Tests for the common code paths located in the test/micro directory. To run the tests in a shell script, follow the instructions in the test/micro/README file. If the target platform supports the use of Tcl (version 8.5 or later), run the Test Suite. How you run the Test Suite varies depending on the target platform: • If the target platform supports a UNIX-like version of Tcl, then set up Tcl and build the Test Suite as described in "Running the Test Suite under UNIX" in Berkeley DB Installation and Build Guide at http://docs.oracle.com/cd/E17076_02/html/ installation/build_unix_test.html and, then, run the test suite. • If the target platform supports a Windows-like version of Tcl, then setup Tcl, and build and run the Test Suite as described in "Running the Test Suite under Windows" in Berkeley DB Programmer's Reference Guide at http://docs.oracle.com/cd/E17076_02/ html/installation/build_win_test.html Reviewing the Results of the Tests It is up to you to submit the results of the tests (test_micro, test_mutex, and, if possible, the entire tcl test suite) for review by Oracle Berkeley DB engineering in order for Oracle to consider providing support for Berkeley DB on a new platform. When Oracle has not agreed to support Berkeley DB on the new platform, you are responsible for ensuring that the tests run successfully. Integrating Changes into the Berkeley DB Source Code Once you have have completed your port, send all code changes that you made as "diff" files to Berkeley DB development for review. Exactly how you integrate changes into the Berkeley DB source code varies depending on whether or not Oracle has agreed to support Berkeley DB on the new platform: At this point, when Oracle has agreed to support Berkeley DB on the new platform: 3/28/2016 1. Berkeley DB development reviews the changes, makes the changes (with modifications if necessary) in the source code, creates another snapshot which includes the new platformspecific changes, and sends you the new snapshot. 2. On receipt of the new snapshot, recompile this new snapshot. If the new snapshot compiles correctly without any changes, test the port again. DB Porting Guide Page 9 Testing and Certifying the Port Library Version 12.1.6.2 Certifying a Port of Berkeley DB When the target platform supports using Tcl, the port is considered certified after a successful standard run (run_std) of the Test Suite . When the target platform does not support using Tcl, a port is considered certified if you see successful message reports for each of the tests in test_micro and test_mutex. Additionally, the configuration and compilation of Berkeley DB must be complete without errors or warninigs of any kind. You must provide specific information about the hardware, software, and compiler used during the porting process, especially versions of all thrid party tools used. If you have other diagnostic tools available, such as memory allocation checking tools, please conduct tests using them as well and provide Oracle engineering with the results. Finally, do not constrain your testing to one configuration. There are many different ways to configure Berkeley DB (that is many options to the "configure" script), please configure, built, and test Berkeley DB on the target platform with as many combinations of these flags as possible to ensure that nothing is missed. Note Contact the Oracle Berkeley DB engineering team for a platform compatibility test suite. 3/28/2016 DB Porting Guide Page 10
Source Exif Data:
File Type : PDF File Type Extension : pdf MIME Type : application/pdf Linearized : No Page Count : 16 Profile CMM Type : Unknown (lcms) Profile Version : 2.1.0 Profile Class : Display Device Profile Color Space Data : RGB Profile Connection Space : XYZ Profile Date Time : 1998:02:09 06:49:00 Profile File Signature : acsp Primary Platform : Apple Computer Inc. CMM Flags : Not Embedded, Independent Device Manufacturer : Hewlett-Packard Device Model : sRGB Device Attributes : Reflective, Glossy, Positive, Color Rendering Intent : Perceptual Connection Space Illuminant : 0.9642 1 0.82491 Profile Creator : Unknown (lcms) Profile ID : 0 Profile Copyright : Copyright (c) 1998 Hewlett-Packard Company Profile Description : sRGB IEC61966-2.1 Media White Point : 0.95045 1 1.08905 Media Black Point : 0 0 0 Red Matrix Column : 0.43607 0.22249 0.01392 Green Matrix Column : 0.38515 0.71687 0.09708 Blue Matrix Column : 0.14307 0.06061 0.7141 Device Mfg Desc : IEC http://www.iec.ch Device Model Desc : IEC 61966-2.1 Default RGB colour space - sRGB Viewing Cond Desc : Reference Viewing Condition in IEC61966-2.1 Viewing Cond Illuminant : 19.6445 20.3718 16.8089 Viewing Cond Surround : 3.92889 4.07439 3.36179 Viewing Cond Illuminant Type : D50 Luminance : 76.03647 80 87.12462 Measurement Observer : CIE 1931 Measurement Backing : 0 0 0 Measurement Geometry : Unknown Measurement Flare : 0.999% Measurement Illuminant : D65 Technology : Cathode Ray Tube Display Red Tone Reproduction Curve : (Binary data 2060 bytes, use -b option to extract) Green Tone Reproduction Curve : (Binary data 2060 bytes, use -b option to extract) Blue Tone Reproduction Curve : (Binary data 2060 bytes, use -b option to extract) Language : en Date : 2016:03:28 12:53:26-07:00 Producer : Apache FOP Version 0.95 PDF Version : 1.4 Create Date : 2016:03:28 12:53:26-07:00 Page Mode : UseOutlinesEXIF Metadata provided by EXIF.tools