You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Monitoring system tests backoff is insufficient sometimes #3739
10 retries is probably too many (after our experience with logging)
I think the issue is a strange combination of list() calls that consume iterators. The retry probably needs to be re-written? (It may predate the rewrite that resulted in the google.cloud.iterator module.)
It might because we cannot read the metric right after it is available to read, and there is a long and unstable cool down period. I think we can have a try to increase the waiting time between each retry by increasing the backoff, and add a RetryErrors decorator to this test function to run again if failed.
Activity
liyanhui1228 commentedon Aug 7, 2017
The max retries for the failed test is 10, but the tests failed after retry for 5 times, what stops it from retrying?
dhermes commentedon Aug 7, 2017
logging)list()calls that consume iterators. The retry probably needs to be re-written? (It may predate the rewrite that resulted in thegoogle.cloud.iteratormodule.)dhermes commentedon Aug 8, 2017
liyanhui1228 commentedon Aug 9, 2017
It might because we cannot read the metric right after it is available to read, and there is a long and unstable cool down period. I think we can have a try to increase the waiting time between each retry by increasing the backoff, and add a
RetryErrorsdecorator to this test function to run again if failed.docs: add samples from bigtable (#38)