Repository navigation
New Cloud Run end-to-end deployment facility #1991
Description
Cloud Run's current facility to deploy end-to-end tests is a bundle of BASH scripts. They seem very prone to flakes, and have generally proven difficult to maintain. #1740 is just the latest case of a flaky test.
A new approach to deploying the samples is needed, one which de-emphasizes BASH. Step 1 I'll put together a proposal.
Activity
fhinkel commentedon Oct 8, 2020
@grayside Thanks! Do you want to keep this as P1?
grayside commentedon Oct 8, 2020
Has Cloud Run flakiness been blocking PRs? If not this can be a P2.
averikitsch commentedon Nov 9, 2020
List of samples to update:
averikitsch commentedon Dec 8, 2020
@grayside for the PubSub sample tests, can I mock the Pub/Sub message by sending a POST request or should I test the user journey by setting up the Pub/Sub topic and sub and checking the logs for the message? The user journey tests seems like it could be flaky due to the need to check the logs.
averikitsch commentedon Dec 14, 2020
All samples have been updated.
averikitsch commentedon Dec 31, 2020
reopening as there are test failures
averikitsch commentedon Jan 6, 2021
The last set of test errors were due to Cloud Build quotas
grayside commentedon Jan 8, 2021
In discussion with Averi, #2124 doesn't fix Cloud Build quotas, so we are seeing two different classes of flakiness. Cloud Build Quotas will be handled in a follow-up.
averikitsch commentedon Jan 14, 2021
Note: There has not been any test failures for the Run samples since the mocha timeouts extension PR was merged. Cloud build quotas have not been an issue, but could be if more Run samples are added or if multiple PRs are testing at the same time.
averikitsch commentedon Feb 4, 2021
Cloud Build flakiness has not been an issue since the record error in December. Closing this issue and will open a new one if tests continue to be flaky.