Skip to content

feat(ppai): modernize ppai weather forecasting - #14764

Open
iennae wants to merge 1 commit into
mainfrom
feat/modernize-ppai-weather-forecasting
Open

iennae wants to merge 1 commit into
mainfrom
feat/modernize-ppai-weather-forecasting

Conversation

@iennae

@iennae iennae commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

Description

Fixes #

Checklist

Testing

  • I have tested this change on a live environment and verified it works as intended.

Compliance & Style


Post-Approval Actions

  • Please merge this PR for me once it is approved

@iennae
iennae requested review from a team as code owners November 5, 2026 15:26
@product-auto-label product-auto-label Bot added samples Issues that are directly related to samples. api: people-and-planet-ai labels Nov 5, 2026
@snippet-bot

snippet-bot Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

Here is the summary of changes.

You are about to add 4 region tags.

This comment is generated by snippet-bot.
If you find problems with this result, please file an issue at:
https://github.com/googleapis/repo-automation-bots/issues.
To update this comment, add snippet-bot:force-run label or use the checkbox below:

  • Refresh this comment

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request modernizes the weather forecasting sample by replacing the custom PyTorch FCN model with a pipeline that integrates Google Earth Engine, BigQuery ML (TimesFM 2.5), Google DeepMind's WeatherNext 3, and Gemini 3.8 Flash for multimodal storm risk nowcasting. The changes include new scripts for data ingestion, time-series loading via Apache Beam, zero-shot forecasting, multimodal analysis, and visualization, alongside updated documentation and comprehensive tests. Feedback on the changes highlights a critical issue in fetch_weather_patch.py where raw netCDF scale factors and offsets are incorrectly applied to already-calibrated GOES thermal IR bands, which would corrupt downstream calculations and visualizations.

@iennae iennae changed the title Feat/modernize ppai weather forecasting feat(ppai): modernize ppai weather forecasting Nov 5, 2026
@iennae
iennae force-pushed the feat/modernize-ppai-weather-forecasting branch 5 times, most recently from 144f964 to e6954b8 Compare November 10, 2026 04:33
… Earth Engine, Apache Beam, BigQuery ML AI.FORECAST, and Gemini 3.8 Flash

Problem:
The legacy people-and-planet-ai/weather-forecasting sample spanned 45 files across four separate notebooks, four duplicate test harnesses, and a custom PyTorch/HuggingFace Cloud Run serving container with checked-in model weights (serving/model/pytorch_model.bin). It relied on the deprecated NASA/GPM_L3/IMERG_V06 Earth Engine collection, hardcoded NOAA/GOES/16/MCMIPF (which ceased operation in April 2025 in favor of GOES-19), used legacy Vertex AI custom training and gcloud AI Platform commands, and included an outdated user feedback form.

Solution:
- Consolidate the multi-notebook workflow into a single end-to-end tutorial notebook (README.ipynb) orchestrating standalone, single-task Python modules aligned with JoTa task_rules.yaml and DevSite Dataflow Python ML references.
- Add fetch_weather_patch.py to query calibrated half-hourly precipitation from NASA GPM IMERG V07 (NASA/GPM_L3/IMERG_V07), cloud-top brightness and moisture bands from NOAA GOES-19 (NOAA/GOES/19/MCMIPF) and GOES-16 (NOAA/GOES/16/MCMIPF), and terrain elevation from MERIT DEM (MERIT/DEM/v1_0_3), featuring Hurricane Melissa (Oct 2025, Jamaica) and Hurricane Helene (Sept 2024, Southern Appalachians) case studies.
- Add load_weather_time_series.py (dataflow_weather_load_time_series) to transform and load hourly satellite observations into BigQuery using Apache Beam.
- Add forecast_precipitation.py (bigquery_weather_forecast_precipitation) for zero-shot multi-hour precipitation nowcasting using BigQuery ML AI.FORECAST with TimesFM 2.5.
- Add analyze_storm_risk.py (googlegenaisdk_weather_analyze_storm_risk) for structured multimodal flood-risk synthesis combining multi-sensor satellite composites and forecasted precipitation curves using the Google Gen AI SDK (gemini-3.8-flash on global).
- Document scaling to global medium-range ensemble forecasts using Google DeepMind WeatherNext 3 in Earth Engine and BigQuery.
- Remove legacy notebooks, serving scaffolding, binary weights, static PNGs, and outdated feedback forms, and add fast offline unit tests in test_weather_forecasting.py.
@iennae
iennae force-pushed the feat/modernize-ppai-weather-forecasting branch from e6954b8 to a273ad8 Compare November 10, 2026 21:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: people-and-planet-ai samples Issues that are directly related to samples.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants