Introduction to BigQuery migration

The BigQuery Migration Service is a comprehensive solution for migrating your data warehouse to BigQuery. It includes features that help you with each phase of migration, including assessment and planning, SQL translation for a variety of SQL dialects, and data transfer.

You can find the BigQuery Migration Service in the BigQuery console when you navigate to Migration > Services.

With the BigQuery Migration Service, you can do the following:

  • Assess: Plan your data warehouse migration by running a BigQuery migration assessment.
  • Translate SQL: The translation services automate the conversion of your SQL queries into GoogleSQL, including Gemini-enhanced SQL customization. You can use the batch SQL translator, the interactive SQL translator, or the SQL translation API to migrate your workloads to BigQuery.
  • Transfer data: Set up a data transfer that loads data from your data source to BigQuery. For more information, see What is the BigQuery Data Transfer Service?.
  • Register or migrate open lakehouse: Migrate metadata from external catalogs, such as Apache Hive Metastore and Apache Iceberg REST Catalogs, to Lakehouse REST catalog tables for Apache Iceberg.

You can also use the following open-source tools to help you with your migration process:

  • Data migration tool: Use the data migration tool to automate your data warehouse migration to BigQuery. The tool uses the BigQuery Data Transfer Service, the BigQuery translation services, and the data validation tool to transfer data, translate and validate DDL, DML, and SQL queries.
  • Data validation tool: After migrating your data to BigQuery, run the data validation tool to validate that your source and destination tables match.
  • BigQuery permission mapper: Use the permission mapper to automate the creation and maintenance of user-modifiable permission maps. You can use the permission mapper to analyze and reconcile duplicate permissions and user groups, while also generating error reports. The tool outputs JSON and Terraform scripts for BigQuery group, user, and binding creation.
  • Managed Service for Apache Airflow templates: Use Managed Service for Apache Airflow templates to simplify the creation of new Airflow DAGs or the migration of existing orchestrated jobs from on-premises to the cloud.
  • Cloud Foundation Fabric: View Terraform examples and modules for Google Cloud, including an organization-wide landing zone blueprint, reference blueprints for network patterns and product features, and a library of adaptable modules.

Migration sources

The BigQuery Migration Service supports the migration of the following data warehouses to BigQuery:

Quotas

Quotas and limits apply to the number of jobs as well as the size of files. For more information on migration service quotas and limits, see Quotas and limits.

Pricing

There is no charge to use the BigQuery Migration API. However, storage used for input and output files incurs the normal fees. For more information, see Storage pricing.

Additionally, you can use the cost estimation functionality in Google Cloud Migration Center to generate a cost estimate of running your data warehouse setup that you migrate to BigQuery. For more information, see Start a cost estimation and Specify data warehousing requirements.

What's next