Dataplex Universal Catalog has evolved into the Knowledge Catalog. Pricing SKUs will remain named as Dataplex. Dataplex pricing is based on pay-as-you-go usage. Dataplex currently charges based on the following SKUs:

• Dataplex Universal Catalog processing (standard and premium)
• Metadata storage

The following is a high-level overview of how each key Dataplex Universal Catalog capability is billed:

Dataplex Universal Catalog standard and premium processing are metered by the Data Compute Unit (DCU). DCU-hour is an abstract billing unit for Dataplex Universal Catalog and the actual metering depends on the individual features you use.

Dataplex Universal Catalog free tier for Knowledge Catalog

As part of the Google Cloud Free Tier, Knowledge Catalog offers some resources free of charge up to a specific limit. These free usage limits are available during and after the free trial period. If you go over these usage limits and are no longer in the free trial period, you will be charged according to the pricing as described in the sections above.

Note: The Knowledge Catalog free tier is only available for the Dataplex Universal Catalog Standard Processing SKU, and is not available for the Dataplex Universal Catalog Premium Processing SKU.

DCU charges for each feature are calculated as follows:

1. Auto data quality scans:

• The DCU-hour consumption is proportional to the processing involved in profiling the data and computing the data quality metrics. This is billed per second, with a minimum of one minute.
• The charge depends on the number of rows, the number of columns, the amount of data that you've scanned, the data quality rule configuration, the partitioning and clustering settings on the table, and the frequency of the scan.
• For data quality anomaly detection scans, DCU charges do not apply. Instead, standard BigQuery pricing for compute, storage, and BQML model training, processing, and deploying apply.
• Using a custom execution identity changes how you are billed for the scan. When you specify a custom execution identity, the compute and storage costs associated with the scan are billed directly to your BigQuery project, bypassing the standard Dataplex Premium SKUs.

2. There are several options to reduce the cost of auto data quality scans:

• Sampling
• Incremental scans
• To separate data quality charges from other charges in the Dataplex premium processing SKU, on the Cloud Billing report, use the label goog-dataplex-workload-type with value DATA_QUALITY.

3. To filter aggregate charges, use the following labels available in billing export in BigQuery:

• goog-dataplex-datascan-data-source-dataplex-entity
• goog-dataplex-datascan-data-source-dataplex-lake
• goog-dataplex-datascan-data-source-dataplex-zone
• goog-dataplex-datascan-data-source-project
• goog-dataplex-datascan-data-source-region
• goog-dataplex-datascan-id
• goog-dataplex-datascan-job-id

4. Data profiling scans:

• The DCU-hour consumption is proportional to the processing involved in profiling the data and computing the data quality metrics. This is billed per second, with a minimum of one minute.
• The charge depends on the number of rows, numbers of columns, the amount of data scanned, partitioning and clustering settings on the table, and the frequency of the scan.
• Using a custom execution identity changes how you are billed for the scan. When you specify a custom execution identity, the compute and storage costs associated with the scan are billed directly to your BigQuery project, bypassing the standard Dataplex Premium SKUs.

5. There are several options to reduce the cost of data profiling scans:

• Sampling
• Incremental scans
• Column filtering
• Row filtering
• To separate data profiling charges from other charges in the Dataplex premium processing SKU, on the Cloud Billing report, use the label goog-dataplex-workload-type with value DATA_PROFILE.

6. To filter aggregate charges, use the following labels available in billing export in BigQuery:

• goog-dataplex-datascan-data-source-dataplex-entity
• goog-dataplex-datascan-data-source-dataplex-lake
• goog-dataplex-datascan-data-source-dataplex-zone
• goog-dataplex-datascan-data-source-project
• goog-dataplex-datascan-data-source-region
• goog-dataplex-datascan-id
• goog-dataplex-datascan-job-id

You pay for the compute power consumed by automatic lineage reporting from supported Google Cloud systems. The billing appears in the project where you ran the job.

To separate data lineage charges from other charges in the Dataplex premium processing SKU, use the label goog-dataplex-workload-type with the value LINEAGE on the Cloud Billing report.

• For more information about how to identify the billing project, see https://cloud.google.com/dataplex/docs/troubleshooting-lineage#project_types.
• For more information about how to access Cloud Billing Reports in the Google Cloud console and how to configure reports, see Analyze billing data and cost trends with Reports | Cloud Billing | Google Cloud Documentation.

The following activities incur charges:

• Automatic lineage parsing: consumes Data Compute Unit (DCU) hours based on the processing complexity. DCUs measure the abstract compute capacity allocated to perform the parsing. Consumption is calculated based on the number of jobs, job executions, and the number of lineage links created. 

You incur these costs when the Data Lineage API Origin sourceType is set to any value other than CUSTOM. Values such as BIGQUERY, DATAPROC, or COMPOSER indicate that the lineage is being automatically extracted by Google Cloud from the service's logs and metadata. This extraction process requires compute resources (DCUs), which is why it is billable. In contrast, CUSTOM indicates that you have performed the parsing yourself and are only storing the result, which is free.

The following activities are free:

• Custom lineage reporting: reporting lineage from third-party systems or custom integrations (where the sourceType is CUSTOM) is free. This includes the following methods:
• Calling ProcessOpenLineageRunEvent, which allows you to import OpenLineage-compatible events.
• Creating a LineageEvent using a custom client to manually report native lineage events.
• Reporting processes and runs: reporting the process metadata itself is free; charges only apply to the automatic parsing of LineageEvents.
• Lineage Consumption: retrieving lineage (using Get or List methods) and deleting lineage (using Delete methods) are free.

In this example, you enable data lineage to track lineage for BigQuery in your project. The project is in the us-central1 location, where one DCU-hour costs $0.089.

Reporting lineage to or from one EntityReference costs approximately 0.0056 DCU-hours. An EntityReference uniquely identifies a data resource within Google Cloud, such as a BigQuery table or a Cloud Storage bucket.

• Daily job: You execute the following BigQuery job once a day for 30 days:

CREATE TABLE tableB1 AS SELECT * FROM tableA JOIN tableC USING (column);

Result: This operation generates lineage links connecting two source tables to one target table (3 distinct entities).

• Every other day job: You execute the following job every other day (15 times a month):

CREATE TABLE tableB2 AS SELECT * FROM tableA, tableC, tableD;

Result: This operation generates lineage links connecting three source tables to one target table (4 distinct entities).

Cost calculation

For this month, assume 1 GiB of data lineage metadata was generated. Costs are calculated as follows:

• Daily job: 30 days * 3 entities * ~0.0056 DCU-hours * $0.089 ~= $0.045
• Every other day job: 15 days * 4 entities * 0.0056 DCU-hours * $0.089 = $0.030
• Storage: (1 GiB - 1 MiB free tier) * $2.00 = $1.998

Total: ~$0.045 + $0.030 + $1.998 = ~$2.07

As users interact with the Knowledge Catalog, API calls are free of charge, including:

• Creating and managing Dataplex Universal Catalog resources
• Creating and managing lineage resources
• Catalog search

This section provides examples of how to calculate the Dataplex Universal Catalog cost.

Small aspects

• User A creates and applies small aspects (1024 bytes each). For $10 per month, the user can store 5 GiB of metadata, which corresponds approximately to 5M aspects. Assuming one aspect per table, this amounts to a total of 5M tables with aspects.
• User B creates 5M aspects of 1 KB each on the 10th of the month, and deletes the aspects on the 20th. The cost is $3.33, calculated as 5 GiB of data divided by one-third month:

Large aspects

• User C creates and applies large aspects (10 KB each). For $10 per month, the user can store 5 GiB of metadata, which corresponds to approximately 500k aspects. Assuming one aspect per table, it amounts to a total of 500k tables with aspects.
• User D creates 10k aspect types (for example ETL, data governance, data quality), and applies large aspects (10 KB each) using each of the 10 aspect types. For $10 per month, the user can store 5 GiB of metadata, which corresponds approximately to 500k aspects. Assuming 10 aspects per table, it amounts to a total of 50k tables with aspects.

This section describes the pricing for Data Catalog, which is in deprecation phase (transition to the Dataplex Universal Catalog in avoid service disruption).

Data Catalog charges apply to metadata storage for Data Catalog and API calls made to the Data Catalog API.

Metadata storage and API call charges accrue daily. You can view unbilled usage on the Google Cloud console.

Note: Pricing models apply to accounts, not projects, unless specified otherwise.

Data Catalog API calls are billed as described in the following table:

Note: Search queries performed on the Data Catalog page in the Google Cloud console are offered at no charge.