Pattern 2: Migration using ETL/CDC

Migration process designed for critical on-line environments, allowing secure roll-back. An ETL/CDC performs the synchronization between both databases. CDC enables real-time parallel, while some ETLs enables nearly-real-time.

Step 1: Model Migration
  1. Using an export or backup of the current DB and LX-LD
 
Step 2: Application Migration
  1. Migrate application: 99% of the SQL is compatible. For incompatibilities:
    • Alternative SQL will be proposed.
    • LeanXcale will be extended to support the functions used.

  2. Migrate Stored Procedure (e.g., PL/SQL): The procedures are proprietary to the current DB (e.g. Oracle PL/SQL). It will translate to an equivalent java process.
 
Step 3: Data migration
  1. Append-only: Append-only or data with an updating timestamp is migrated. They can be read from backups to avoid overload problems in the production database. LX Loader or ETL (e.g. AWS Glue, Datafactory or Airbyte)
  2. Last Day: Last synchronization of the last period (typically the last day) and data without an updating timestamp. It can be performed during an idle window. LX Loader or ETL (e.g. AWS Glue, Datafactory or Airbyte)
 
Step 4: Parallel
  1. Original Database as Reading Master and an ETL/CDC: Parallel with the original database as the Master Reader. An ETL/CDC populastes changes from the Master database to LeanXcale.
  2. Reading in LeanXcale and CDC: Parallel with LeanXcale as Read Master. Upgrade the application and use ETL/CDC to populate the changes from the original database to LeanXcale.
  3. Original Database deactivation: LeanXcale remains as the only engine.
 
Step 5: Optimization
  1. Application optimization to take advantage of LeanXcale capabilities (parallelization/multithreading).