Repository navigation
Remove Storage Intelligence and Storage Insights resources after GCS optimizations #368
Description
Storage Intelligence and Storage Insights were set up in the httparchive project in August 2026 for a GCS cost analysis. Once the resulting optimizations for gs://httparchive (storage class tiering, lifecycle rules, bucket location) are decided and implemented, remove the resources below. None of them are managed in Terraform.
Why
- Storage Intelligence is billed at Standard edition. The 30-day trial expired on 2026-09-09 and the project's effective edition is now
STANDARD. The object management fee is about $2.50 per million objects per month; with ~900M objects ings://httparchivethat is roughly $2,250/month. Check the Cloud Storage lines in billing to confirm the actual charge. - The dataset has had no new data since 2026-08-23, so the subscription currently produces nothing usable.
Resources
| Resource | Location | Notes |
|---|---|---|
Storage Intelligence config (projects/226352634162/locations/global/intelligenceConfig) |
global | Edition STANDARD |
Insights dataset config storage_intelligence |
us-central1 |
CONFIG_STATE_ACTIVE, 90-day retention |
Linked BigQuery dataset storage_intelligence_ce5cb84c_8368_4246_9a23_066f9724d247 |
us-central1 |
Read-only; removed with the link |
Inventory report config 336706ed-d933-4e0d-943c-fca8340675e2 (gs://httparchive) |
us |
End date 2026-08-25 |
Inventory report config ea73c81d-bef0-41a9-8799-bdc542d5a372 (httparchive.appspot.com) |
us-east4 |
Destination bucket already deleted |
Inventory report config b368ce9a-0509-4b82-8b7f-6cd78e6371b6 (staging.httparchive.appspot.com) |
us-east4 |
Destination bucket already deleted |
The report destination buckets (httparchive-gcs-insights-reports-us, httparchive-gcs-insights-reports-us-east4) are already gone.
Cleanup
Do this only after the analysis is no longer needed. Deleting the link removes the BigQuery dataset and its history.
gcloud storage insights dataset-configs delete-link storage_intelligence --location=us-central1 --project=httparchive
gcloud storage insights dataset-configs delete storage_intelligence --location=us-central1 --project=httparchive
gcloud storage insights inventory-reports delete projects/226352634162/locations/us/reportConfigs/336706ed-d933-4e0d-943c-fca8340675e2
gcloud storage insights inventory-reports delete projects/226352634162/locations/us-east4/reportConfigs/ea73c81d-bef0-41a9-8799-bdc542d5a372
gcloud storage insights inventory-reports delete projects/226352634162/locations/us-east4/reportConfigs/b368ce9a-0509-4b82-8b7f-6cd78e6371b6
gcloud storage intelligence-configs disable --project=httparchiveDone when
- GCS optimizations for
gs://httparchiveare decided and implemented - Storage Intelligence is disabled, and the dataset config, link, and report configs are deleted
- The Storage Intelligence fee no longer appears in billing
Activity
max-ostapenko commentedon Oct 29, 2026
GCS optimization options for
gs://httparchiveFindings from the Storage Intelligence data (inventory snapshot of 2026-08-22, activity 2026-08-11 to 08-23) and the bucket's current lifecycle rules. All figures are at list prices; actual billing may differ.
Current setup:
results/(~4.6 PiB) moves to Coldline at 30 days and is deleted at 400 days.crawls/is deleted at 40 days, so there's nothing to tier there. Soft delete is off. Reads are ~0.8 TiB/month. Writes are ~1 PiB/month, coming from crawlers in 7 US regions.results/(change the target class of the 30-day rule)$3.6k/month in higher transition charges (Archive Class A $1.00 vs Coldline $0.40 per 10k), so **$18k net**results/is read after 30 daysOption 1: Archive for
results/results/read after 30 days, for example by reprocessing or backfills? Storage Intelligence object events didn't capture this bucket's reads, so the data can't answer it. If nothing reads it, this is a one-rule lifecycle change.Option 2: single region
86% of writes come from regions other than the busiest one (us-east1). Unless the crawlers write from a single region, inter-region transfer ($17k/month) cancels most of the replication saving.gs://httparchivepaths.Retention
Keeping
results/for 400 days is the biggest cost driver. Each month of retention is0.35 PiB of Coldline ($2.5k/month). If 400 days is more than the project needs, shortening it is the simplest saving.Suggested order
results/after 30 days. If it isn't read, switch the 30-day rule to Archive.