Hey there ! If you need to diff two Cloud Composer environments, this tool is for you.

environment_diff is a python command line tool to diff two Cloud Composer environments.

As per the example above, this tool looks at various Cloud Composer environment attributes and diffs them. It shows which attributes have different values across environments, and what those values are. It also shows which attributes have a common value across environments, and what this common value is.

The tool can compare the follow attributes of Cloud Composer environments:

• Cloud Composer image version
• Airflow configuration overrides
• PyPI packages
• Environment variables

When comparing two Cloud Composer environments that share no common attribute values the "matching_value" column will be empty as per the example below:

When comparing two Cloud Composer environments that share common attributes values the "matching_value" column will be populated and the "env_1_value" and "env_2_value" columns will be empty. This is shown in the example below.

• Python version: application has been tested on python version 3.10
• Composer version: compatible with both Composer 1 and Composer 2
• Composer version: compatible with all Composer configurations
• GCP IAM Permissions: executing user needs composer.environments.get permission

Clone the code from GitHub to your local machine

navigate to the environment_diff directory

from within the environment_diff directory run the following command

pip install .

Install gcloud

Select a user which has access to both Cloud Composer environments to compare. This user needs composer.environments.get permission

With this user account, initialize gcloud:

gcloud init

Next, authorize the application to access Cloud Composer environments:

gcloud auth application-default login

environment_diff is a command line tool.

Details about expected inputs can be gathered by running the snippet below.

python -m environment_diff --help

To compare two environments the project ID, location & name of each composer environment need to passed as input into the tool.

python -m environment_diff \
--env1_project_id YOUR_PROJECT_ID \
--env1_location YOUR_ENV1_LOCATION \
--env1_name YOUR_ENV1_NAME \
--env2_project_id YOUR_PROJECT_ID \
--env2_location YOUR_ENV2_LOCATION \
--env2_name YOUR_ENV2_NAME

Upon executing the command, the tool will make APIs to Cloud Composer to fetch details of the two specified environments. The details are then compared and the result diff is sent back to the requesting user and printed out to the console

It is also possible to output the detected differences in a JSON file. The file is named "environment_diff.json". To request a JSON file, an additional flag is required as per the example below.

python -m environment_diff \
--env1_project_id YOUR_PROJECT_ID \
--env1_location YOUR_ENV1_LOCATION \
--env1_name YOUR_ENV1_NAME \
--env2_project_id YOUR_PROJECT_ID \
--env2_location YOUR_ENV2_LOCATION \
--env2_name YOUR_ENV2_NAME \
--json_output