Deleting a managed instance

Setup

Clone or navigate to the sourcegraph/deploy-sourcegraph-managed repository

1. Navigate into the repository

• cd deploy-sourcegraph-managed

1. Pull in the latest changes

• git checkout main
• git pull

1. Create a branch for the teardown

• git checkout -b $CUSTOMER/destroy-managed-instance

1. Setup the environment:

• export TF_VAR_opsgenie_webhook=<value from 1Password>
• export CUSTOMER=<customer>

Navigate to the customer’s managed instance directory

cd $CUSTOMER

Allow the GCP KMS Crypto Key to be deleted

sed -i '' 's/    prevent_destroy = true/    prevent_destroy = false/g' infrastructure.tf

By default the KMS Crypto Key is prevented from being deleted. This must be changed in order for Terraform to remove all resources. Without this change you will see the following error:

Error: Instance cannot be destroyed

on infrastructure.tf line 539:

539: resource “google_kms_crypto_key” “key” {

Resource google_kms_crypto_key.key has lifecycle.prevent_destroy set, but the plan calls for this resource to be destroyed. To avoid this error and continue with the plan, either disable lifecycle.prevent_destroy or reduce the scope of the plan using the -target flag.

Destroy the infrastructure

This will remove all GCP infrastructure except the Terraform remote state GCP bucket associated with the customer, which is handled in the next step.

terraform destroy

Disable ‘prevent_destroy’

The default behavior of a GCP bucket is to prevent the destruction of the bucket through the API. It is necessary to remove this protection.

cd gcp-tfstate
sed -i '' 's/resource "google_storage_bucket" "tfstate" {/resource "google_storage_bucket" "tfstate" {\nforce_destroy = true/g' gcp-tfstate.tf
terraform apply

Once force_destroy has been set to “true”, Terraform is able to remove the GCP Bucket.

terraform destroy

Prepare the changeset

cd ../ # to deploy-sourcegraph-managed/$CUSTOMER
git add . && git commit -m “${CUSTOMER}: change flags to force resource removal”
cd ../ # to deploy-sourcegraph-managed
git rm -r $CUSTOMER && git commit -m “${CUSTOMER}: remove managed instance infrastructure”
git push origin HEAD

Create the Pull Request

Title: CUSTOMER: Teardown Managed Instance Link tear-down request ticket in the description

Checkout the sourcegraph/infrastructure repository and ensure it is up-to-date

git checkout main
git pull

Create a new branch

git checkout -b managed-instance/${CUSTOMER}/teardown-project

Remove project variable

cd gcp/projects

Edit terraform.tfvars to remove the project variable corresponding to $CUSTOMER.

Remove the GCP project

terraform apply

Review the proposed changes carefully.

Commit the change

To prevent an initial state circular depenency, the Terraform state for GCP projects is committed as a file into the infrastructure repository. Make sure to include this in the pull request.

git add terraform.tfvars terraform.tfstate
git commit -m "managed-instance-${CUSTOMER}: Remove GCP project"

Open a pull request

git push origin HEAD

For the Pull Request: Title: managed-instance-customer: Remove GCP Project Link tear-down request ticket in the description

Wait for checks to pass, approval and then merge pull request.

Checkout the sourcegraph/infrastructure repository and ensure it is up-to-date

git checkout main
git pull

Create a new branch

git checkout -b managed-instance/${CUSTOMER}/teardown

Remove the DNS entry

cd dns

Edit sourcegraph.managed.tf to remove the cloudflare_record resource for $CUSTOMER.

Commit the change

git add sourcegraph.managed.tf
git commit -m "managed-instance-${CUSTOMER}: Remove DNS entry"
cd ../ # back to the infrastructure/ repo root directory

Open a pull request

git push origin HEAD

For the Pull Request: Title: managed-instance-$CUSTOMER: Remove DNS entry Link tear-down request ticket in the description

Wait for checks to pass, approval, and merge pull request.

Update the local main branch

git checkout main
git pull

Remove the DNS the change

Check .tool-versions for correct Terraform version

cd infrastructure/dns
cat .tool-versions
terraform apply
cd ../ # back to the repo root directory

Remove the GCP project

cd infrastructure/gcp/projects
cat .tool-versions
git checkout main
git pull
terraform apply
cd ../ # back to the repo root directory

Updating the upgrade template

Remove customer from managed instance upgrade template. Avoid mentioning the customer name directly, the handbook is a public repository.

The template is located in handbook/content/departments/product-engineering/engineering/process/releases/upgrade_managed_issue_template.

Updating outstanding tracking issues

Search for any open upgrade tracking issues, edit the description to remove the customer’s entry. No need to upgrade if there isn’t anything to upgrade!

This Github Query may be helpful.