What is data migration? | Definition and common processes

What is data migration?

Data migration is the process of selecting, preparing, and moving existing data from one computing environment to another. Data may be migrated between applications, storage systems, databases, data centers, and business processes.

Each organization’s data migration goals and processes are unique. They must consider many factors such as costs, timing, technical requirements, impact to business operations, the potential for data loss, compliance requirements, and more.

Why do businesses migrate data?

Businesses may undertake data migration projects for a number of reasons, such as to:

• Reduce media, storage, or other IT equipment costs
• Expand and scale storage capacity
• Improve customers’ website or digital experience
• Centralize and simplify data management
• Accelerate application performance
• Merge data from a company acquisition
• Meet new compliance or security requirements
• Enhance data analytics and reporting capabilities

Consider the following example: when someone buys a new computer, they usually prefer to install the newest versions of software and just copy over the most important files from their old one. Adding a bloat of obsolete software and files would unnecessarily take up storage space and slow down their new device. Likewise, efficient data migration ensures that the new system is utilizing correctly cleansed, extracted, and transformed data.

Data migration can be a key enabler of digital transformation — the use of digital technology to modernize business workloads and processes. It often goes hand-in-hand with cloud migration — specifically to ensure that no outdated or corrupt data is migrated to an organization’s new cloud infrastructure.

What are the main types of data migration?

Data centers store files or databases used by software applications, which drive business processes and workflows. As such, data migration is commonly categorized into five types:

1. Storage migration transfers data from one storage medium to another. Organizations may change physical media formats (such as from paper to digital files or hard disk drives), or from on-premise storage to cloud storage. Data can also be migrated between one or more cloud storage systems. After a storage migration, how the data is accessed changes, although the data itself does not.
2. Application migration shifts software applications from one computing environment to another. This may include migrating application programs from an on-premise server to a cloud environment, between clouds (e.g., from AWS to Microsoft Azure), or upgrading an application and retiring the old one. Because every application has a unique data model, the format of the data (as well as how it is viewed by end users) may change during an application migration.
3. Business process migration transfers applications or databases (such as a CRM or ERP platform) that are operated by humans to produce a service for customers. A business process migration is usually prompted by a company merger, acquisition, or reorganization.
4. Database migration, also sometimes called schema migration, moves data between two or more databases. Databases are managed with database management systems (DBMS) such as Oracle, MySQL, PostgreSQL, and others, so database migration can mean moving from one DBMS to another, or upgrading to a newer DBMS version.
5. Data center migration refers to transferring assets from one data center to another location or operating environment. A data center migration is particularly complex because data centers include IT assets that store, retrieve, distribute, or archive data and applications. Depending on the organization’s objectives, data center migration can involve completely changing physical hardware, virtual machines, or cloud solutions.

What does the data migration process involve?

There is no “one size fits all” process for every type of data migration. However, a complete data migration plan contains three phases, which then comprise a number of other components and stages.

1. Pre-migration
2. Migration (“go-live”)
3. Post-migration (test/audit)

Pre-migration (planning/discovery)

Pre-migration is the initial planning phase, which ensures that the migration will go smoothly and aims to minimize risks. During this phase, the data migration teams establish project objectives, scope, staffing/resources required, and critical requirements.

Pre-migration tasks can include (but are not limited to):

• Assessing (profiling) data sources, destinations and formats
• Inspecting data quality, anomalies, or duplications
• Identifying impacted users and potential disruption
• Defining hardware, software, and security requirements
• Determining costs, staff, and data migration tools required
• Setting a migration completion timeline
• Cleansing or reformatting data
• Backing up data and determining what to do with obsolete data
• Deciding on specific approach (described in the next section)
• Creating risk mitigation and stakeholder communication plans

Migration (“go-live”)

Once the plan has been created, the right permissions are secured, and all the data is ready for migrating to the target system, the actual data migration begins. The “go live” execution can include:

• Loading the necessary permissions and settings
• Testing the migration with a mirror of the live environment
• Implementing the data migration policies and security rules
• Testing data in the new system to ensure it is accurate
• Fixing problems from migration

There are a few specific strategies for application migrations to the cloud, such as re-hosting (also called “lift and shift”), re-architecting, re-platforming, and others. Read “What is cloud migration?” to learn more.

Post-migration (validation)

Data migration is not complete after “flipping the switch.” The results of the migration must be audited and validated to make sure everything has been correctly transferred and logged.

Once the post-migration audit is deemed successful, the old system can be decommissioned.

Common data migration approaches

In a data migration, organizations may decide to migrate one system to another one or merge two systems into a brand new one. Whichever strategy they choose, there are two common data migration approaches.

"Big bang" migration: The “big bang” data migration moves an entire dataset in one phase from the legacy system to the new target system. This migration typically occurs over or during a planned downtime period (such as a weekend or holiday).

Pros:

• Requires a shorter implementation timeline
• Users do not need to switch back and forth between two different systems
• Potentially lower cost since both systems do not need to be maintained simultaneously

Cons:

• Increases the burden on the initial planning, development, and testing phases
• Users have little time to get familiar with the new system
• Failures in one part of the system can cause problems in others — requiring a complete rollback to the old system

"Trickle-feed" migration: Global businesses or critical infrastructure providers that need to avoid periods of data inaccessibility may pursue a “trickle-feed” data migration, instead. This phased approach packages and transfers data in smaller increments.

Pros:

• Can happen virtually anytime
• Gives end-users more time to learn and adapt to the new system
• Failures in one part of the system have less impact to the entire organization

Cons:

• Requires more synchronization across the organization
• Potential end-user confusion about switching back and forth between systems
• Potentially higher costs from running both the old and new system for a period

Data migration vs. data integration

Data migration and the term “data integration” are sometimes used interchangeably, but they are distinct. Unlike a data migration project — which only happens once — data integration is an ongoing process involving incremental data changes. Also, unlike data migration, data integration can combine data residing in different locations into one unified view.

Data migration can be a key milestone for data integration initiatives. For example, a business may migrate large amounts of unstructured data (such as music, video, and images) to a new object storage service, but integrate that data with a media processing tool hosted elsewhere.

Can Cloudflare help with data migration?

Cloudflare R2 is an AWS S3-compatible, globally distributed object storage that allows developers to store large amounts of unstructured data with zero egress fees. R2 enables customers to quickly and easily migrate data objects stored in other cloud providers to an R2 bucket of their choice, with the R2 Migrator (also known as Super Slurper).

Cloudflare D1, an SQL database at the edge, also enables developers to import existing SQLite tables and their data directly.

Learn more about Cloudflare's developer platform.