<?xml version='1.0'?>
<!DOCTYPE book PUBLIC "-//OASIS//DTD Docbook XML V4.1.2//EN" "http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd">
<chapter id="node-1138">
<chapterinfo>
<releaseinfo>
md5-hash:c5c0618c62b70d44e0f6fdc51d75528f
weight:-14
depth:2
</releaseinfo>
</chapterinfo>
<title>Overview</title>
<literallayout><![CDATA[Plenty of software uses a database to store data. There are manytypes of database implementations but generally two main branches exist -SQL based relation database management systems (RDBMS) and file based database.

<p cc="cc" />One of the RDBMS is an Interbase.The Interbase is powerful transaction SQL database engine that implements many advancedfeatures - views, triggers, stored procedures, generators, etc. Interbasewas developed by Borland as commercial database but version 6.0was released as open-source. It means that it's possible use itfor-free for any purposes. On the open-source database was foundednew branch named Firebird. Even Borland released next Interbase v7as non open-source, the Firebird clone remains open-source.

<p cc="cc" />Even the Interbase architecture is robust and powerful, comparable to other concurrent,one feature limits it - there is not implemented replication engine such asin Oracle, MSSQL or even MSAccess.

<p cc="cc" />In many cases data of an enterprise are not stored in one database butthey are distributed to more locations. Replication engine try to maintaindata in each of databases as synchronous as possible. To be replicationefficient, the engine catches only data changes and passes them to the other databases.

<p cc="cc" />There are many reasons why to split data:<ul compact id="opus"><li />non-reliable connection to main database<li />security, shadow database in DM zone<li />data backup to remote <em id="opus">shadow</em> database at different place<li />external database on laptops for representatives<li />etc.

<p cc="cc" /></ul >There are many replication models because two enterpriseswill have probably different requirements.

<p cc="cc" />Lets see several examples.

<p cc="cc" />There are main database and shadow (read-only) database that servesas backup. When the main database crashes, <em id="opus">shadow</em> database is activatedas main one.

<p cc="cc" />There are more equivalent databases. Data should be the same in each of them.

<p cc="cc" />Enterprise maintains net of agents that have database at personal laptops. Theycan change records (make orders, update information). And because they arecompetitors they can see only own data.

<p cc="cc" />Enterprise must consolidate data daily from subsidiaries.

<p cc="cc" />So there are many questions:<ul compact id="opus"><li />how often replicate?<li />does exist reliable and fast online connection?<li />replicate all tables?<li />replicate all records in table?<li />uni or bidirectional?

<p cc="cc" /></ul >Because replicator must meet all contradictory requirements,changes fire triggers that log them in source database and then they are replicatedto targets. It means that are transferred only changes. Due toreferential integrity it's necessary replicate in the same sequentialorder as the changes occurred. If new records may be insertedin more locations, primary keys must be unique -every record must be identifiable by unique primary key.The logging brings overhead and the source database growths, so replicationlog records are deleted when are successfully replicated.You can configure what fields values log - generally primary key values must be logged mandatory,<em id="opus">NOT NULL</em> and foreign key fields recommended and remaining values optionally.

<p cc="cc" />Let's suppose that the replicator connects to the source database, looksto change log and if there are some records connects to the target database andtransfers the changes. One replicator is connected to both databases in one time,hence it's <b id="opus">online replication</b>.But what happen if connection is broken beforetransfer is finished? Nothing dangerous, change log record is removedwhen change is committed in target database - it simulates two phase commit.But when an constraint problem occurs record is moved to other log whereare stored conflicting records. The replicator cannot solve such problemautomatically, it must be solved manually, hence the log is named <b id="opus">manual log</b>.

<p cc="cc" />How is possible replicate when one of databases is unavailable from oppositelocation? It is hidden behind the firewall, machine where is running connectsvia dial-up or connection is unreliable. The solution is <b id="opus">offline replication</b>.In this case the first replicator is connected to source database and preparespackages containing replication data and passes them to a shared drop-off location where are leftuntouched until the second replicator reads the packages fromthe shared location and applies data to the target database.The offline replication uses robust four-phase acknowledge protocol andstores package transfer history if <b id="opus">transfer log</b>.

<p cc="cc" />In ideal world all data are correctly replicated. But in real worldfrom time to time happen an exception or an accident (hard disk crashes, replicationintegrity corrupts, etc.). Even no crash happen there is one exception -start of replication, because you must setup data to be synchronousbefore replication is done. Who can manage databases to be ready forreplication? Answer is a <b id="opus">synchronization</b>. The replication transfersonly changes whereas the synchronization compares databases, table to table,record to record. This process is of course much more time consumingespecially for large databases. Note that synchronization is always online.

<p cc="cc" />Now let's suppose that there is an table containing very importantdata and somebody intentionally rewrites it. This problem solves<b>history</b> - replicator keep record history in <b id="opus">history log</b>. Unlikethe replication the history is not replicated to other database.

<p cc="cc" />Another possibility how to keep record history is <b id="opus">incremental SQL log</b>.It's a plain text file containing sequential list of SQL commands. Note evenInterbase data manipulation language does not support BLOB values, the incremental logcontains also BLOB metadata. Since it is plain SQL text file it can be usedto feed changes to any SQL database, e.g. you can replicate changes to MySQL.Both the online and the offline methods are supported.

<p cc="cc" />If data are changed at both sides data inconsistency can occur,when corresponding record is changed in the source database andthe target database. A <b id="opus">field/record conflict</b> happened.The replicator is able to recover conflict automatically accordingor to pass conflict to the manual log. The recovery isconfigurable for every relation.

<p cc="cc" />All the configuration data, i.e. schemes are stored to a separate<b id="opus">configuration database</b>. You can prepare it once and use for alllocations, does not matter if you access it online or make physical copy.It simplifies administration significantly.<h2 id="opus">Features</h2><ul compact id="opus"><li />the Interbase and Firebird (1.0, 1.5) databases supported<li />the SQL Dialect 1 and 3 supported<li />the offline or online synchronization<li />FTP, email or shared net dir transfer for offline replication, open source transfer libraries<li />the offline packages can be encoded for improved security or compressed, open source interface, CRC checking<li />visual replication manager (scheme editor)<li />custom configurable history of records<li />the replication server running as a GUI application or a console/NT service application<li />multi-threading supported<li />multi-segment primary key supported<li />conditional replication and synchronization<li />extended conflict checking<li />replication into tables or stored procedures<li />replication actions and errors are logged into logs<li />replication statistics<li />international support - optional UTC time stamps, Unicode/UTF-8 encoding<li />source/target database recovery from daily backup and incremental backup<li />SDK - a developer can implement full replication server functionality to own software<li />password encryption<li />database backup/restore<li />database cloning<li />replication of external files (linked from database using file name)

<p cc="cc" /></ul ><h2 id="opus">Features not yet implemented</h2><ul compact id="opus"><li />replication of array fields (not planned)<li />not tested for Asian code pages<li />replication of structure (DDL)<li />record history viewer and SDK<li />GUI driven database comparer (dbdiff)<li />custom installer NSIS support

<p cc="cc" /></ul ><h2 id="opus">Requirements</h2><ul compact id="opus"><li />installed Interbase/Firebird client<li />the SQL server must support <em id="opus">CommitRetaining</em> and <em id="opus">RollbackRetaining</em> (Interbase v6)<li />Windows 9x/ME/NT/2000/XP or Linux at x86 platform</ul >]]></literallayout>
</chapter>
