Technical Bulletin: GT.M Database Endian Conversion - UNIX
Copyright © 2006 Fidelity National Information Services, Inc.
June 14, 2006
| Revision History | ||
|---|---|---|
| Revision 1.0 | 14 June 2006 | Introduced the MUPIP ENDIANCVT command. |
| Revision 1.1 | 19 March 2008 | Added rules for migrating database data between platforms. |
|
|
GT.M supports platforms with both little endian and big endian CPU architectures.
- Little endian: GNU/Linux on x86, x86_64 and Itanium; Tru64 UNIX and OpenVMS on Alpha/AXP
- Big endian: AIX on pSeries, HP-UX on Itanium and PA-RISC; Solaris on SPARC
GT.M database files contain binary data stored in the native endian format of the system on which it is created and used. To migrate data from to a platform with a different endian architecture in prior versions of GT.M, one had to export data into ASCII format with a MUPIP EXTRACT -FORMAT=ZWR on the source platform and import with a MUPIP LOAD on the target platform. The procedure for large databases can be time consuming, although it can be parallelized with multiple concurrent extract and load procedures. The ASCII extract file can also be very large, necessitating adequate temporary storage. Starting with V5.1-000, GT.M introduces a new MUPIP ENDIADCVT command on UNIX/Linux platforms that performs an in-place endian conversion on database files that eliminates the need for intermediate storage and potentially speeds up the migration. Use the following rules when migrating database data between platforms:
- Between one UNIX/Linux platform and another with the same endian-ness: simply copy the database files (if processes are actively using the database, use MUPIP BACKUP). Note: The GT.M version first used on the target to update the database must be exactly the same as that on which it was last updated on the source.
- Between one UNIX/Linux platform and another with a different endian-ness: use MUPIP ENDIANCVT on the source before copying the database file or on the target after copying it. Note: The GT.M version used for the MUPIP ENDIANCVT must be exactly the same as that last used to update the database.
- Between an OpenVMS platform and any UNIX/Linux platform: MUPIP EXTRACT on the source platform followed by a MUPIP LOAD on the target platform.
![[Note]](images/note.png)
Since GT.M on OpenVMS does not support Unicode, please do not attempt to migrate Unicode data to OpenVMS.
mupip endiancvt [-outdb <outdb-file>] <db-file>
The command converts a database file from one endian format to the other (BIG to LITTLE or LITTLE to BIG).
Requirements:
ENDIANCVT requires standalone access to the database
An informational message with the “from” and “to” endian formats is printed as is a confirmation request to perform the conversion. Conversion begins only upon receiving positive confirmation, which is a case insensitive "yes".
<db-file> is required and is the source database for endian conversion. The default is to convert <db-file> in place.
-outdb : Write the converted output to <outdb-file>. In this case, the source database <db-file> is not modified.
<outdb-file> will be exactly the same size as <db-file>. Adequate storage is required for <outdb-file> if endian conversion is to complete successfully.
GT.M on a big endian platform can convert a little endian database into big endian and vice versa; as can GT.M on a little endian platform. Standalone access is required to perform conversion. GT.M (run-time and utilities other than MUPIP ENDIANCVT) on a given endian platform will only open and process databases that are in the same endian format. An attempt to open a database of a format other than the native endian format triggers an error and the database open is aborted.
The source and target platforms for ENDIANCVT must both be UNIX. MUPIP ENDIANCVT does not convert database files created on OpenVMS to UNIX format and vice versa. For transferring OpenVMS database content to UNIX and vice versa, the EXTRACT/LOAD approach must be used. MUPIP ENDIANCVT is not supported on OpenVMS as GT.M databases on OpenVMS are always in little endian format.
MUPIP ENDIANCVT converts database files only. Other files such as global directory, binary extract, and journal files must be created in the native format on the target platform.
Before running endian conversion, ensure that the database does not have any integrity errors. An existing error may be propagated or even exacerbated by endian conversion. Journaling (and replication) must be disabled before performing endian conversion.
If converting in-place, having a backup is strongly recommended since, if the conversion does not complete for any reason, including a software bug, the database will not be recoverable.
The output of DSE DUMP -FILEHEADER now includes the endian format of the database file as follows:
DSE> dump -fileheader
File /gtm/V51000/mumps.dat
Region DEFAULT
Date/Time 01-JUN-2006 08:59:46 [$H = 60409,32386]
Access method BG Global Buffers 1024
Reserved Bytes 0 Block size (in bytes) 1024
Maximum record size 256 Starting VBN 129
Maximum key size 64 Total blocks 0x00000065
Null subscripts NEVER Free blocks 0x00000062
Standard Null Collation FALSE Free space 0x00006000
Last Record Backup 0x0000000000000001 Extension Count 100
Last Database Backup 0x0000000000000001 Number of local maps 1
Last Bytestream Backup 0x0000000000000001 Lock space 0x00000028
In critical section 0x00000000 Timers pending 0
Cache freeze id 0x00000000 Flush timer 00:00:01:00
Freeze match 0x00000000 Flush trigger 960
Current transaction 0x0000000000000001 No. of writes/flush 7
Maximum TN 0xFFFFFFFFDFFFFFFF Certified for Upgrade to V5
Maximum TN Warn 0xFFFFFFFF5FFFFFFF Desired DB Format V5
Master Bitmap Size 64 Blocks to Upgrade 0x00000000
Create in progress FALSE Modified cache blocks 0
Reference count 1 Wait Disk 0
Journal State DISABLED
Mutex Hard Spin Count 128 Mutex Sleep Spin Count 128
Mutex Spin Sleep Time 2048 KILLs in progress 0
Replication State OFF Region Seqno 0x0000000000000001
Zqgblmod Seqno 0x0000000000000000 Zqgblmod Trans 0x0000000000000000
Endian Format LITTLE
|
DBENDIAN |
Database file xxxx is aaaa endian on a gggg endian system |
|
Severity: |
Error |
|
MUPIP/Run Time Error: |
This indicates that the database file being opened is in the wrong endian format for the current system. This usually means that the file was copied from another system with the opposite endian format. |
|
Action: |
To use the database file on the current system, change the endian format using the MUPIP ENDIANCVT command. |
|
NOENDIANCVT |
Unable to convert the endian format of file xxxx due to yyyy |
|
Severity: |
Error |
|
MUPIP Error: |
One of the requirements for the MUPIP ENDIANCVT command was not met. The problems reported are: “database format is not the current version”, "minor database format is not the current version”, "some blocks are not upgraded to the current version”, "kills in progress", "the database is frozen", "a GT.CM server accessing the database", "recovery was interrupted", "database creation in progress", "wc_blocked is set - rundown needed", "the database is corrupted". |
|
Action: |
Resolve the reported conditions and repeat the command. |
|
ENDIANCVT |
Converted database file xxxx from yyyy endian to zzzz endian on a wwww endian system |
|
Severity: |
Informational |
|
MUPIP Information: |
When MUPIP ENDIANCVT has successfully completed, it displays the conversion information. This information includes the database file, its previous endian format, the new endian format, and the endian format that is native to the current system. |
Command Syntax: UNIX syntax (i.e., lowercase text and "-" for flags/qualifiers) is used throughout this document.
Reference Number: The reference numbers used to track software enhancements and customer support requests appear in parentheses ( ).
Platform Identifier: If a new feature or software enhancement does not apply to all platforms, the relevant platform appears in brackets [ ].