[tehut]

Bug:
Support informed us of customers running into counterintuitive behavior where deployments were failing their progress deadline while paused.

Fix:

• Updated the deploymentwatcher.shouldFail function not to fail deployments if their status is "paused"

The changes below are intended make sure deployments don't fail as soon as they're un-paused

• Updated the deploymentwatcher.getDeploymentStatusUpdate function to set an UpdatedAt time if the new deployment status is "paused" or "running" (might make sense to only reset on running instead of both)
• Update the state store's updateDeploymentStatusImpl function to overwrite the deployment's ProgressDeadline with the UpdatedAt time + ProgressDeadline duration if neither u.UpdatedAt nor u.ProgressDeadline are 0.

Testing & Reproduction steps

• Added an additional test case to deployments_watcher_test.go to assert:
  - that a paused deployment past its progress deadline will not fail
  - that the deployment's progress deadline == UpdatedAt + ProgressDeadline

Links

ref: https://hashicorp.atlassian.net/browse/NMD-1080?atlOrigin=eyJpIjoiMDI5YTFlZjZiMGM2NGQ4MWIwMDU1NjBhZDFjNzFiNjciLCJwIjoiaiJ9

Contributor Checklist

• Changelog Entry If this PR changes user-facing behavior, please generate and add a
  changelog entry using the make cl command.
• Testing Please add tests to cover any new functionality or to demonstrate bug fixes and
  ensure regressions will be caught.
• Documentation If the change impacts user-facing functionality such as the CLI, API, UI,
  and job configuration, please update the Nomad product documentation, which is stored in the
  web-unified-docs repo. Refer to the web-unified-docs contributor guide for docs guidelines.
  Please also consider whether the change requires notes within the upgrade
  guide. If you would like help with the docs, tag the nomad-docs team in this PR.

Reviewer Checklist

• Backport Labels Please add the correct backport labels as described by the internal
  backporting document.
• Commit Type Ensure the correct merge method is selected which should be "squash and merge"
  in the majority of situations. The main exceptions are long-lived feature branches or merges where
  history should be preserved.
• Enterprise PRs If this is an enterprise only PR, please add any required changelog entry
  within the public repository.

• If a change needs to be reverted, we will roll out an update to the code within 7 days.

Changes to Security Controls

Are there any changes to security controls (access controls, encryption, logging) in this pull request? If so, explain.

[gulducat]

LGTM! I have only one nit to pick.

[gulducat]

not your addition, and this is a super nit, but: it feels a bit odd to return fail immediately after its assignment to false. I more expect its assignment before a situation where it might change.

we don't even really need it at all, since it's a named return whose default zero value is false, but if we want to keep it, then I vote:

Suggested change

	fail = false
	if d.Status == structs.DeploymentStatusPaused {
	return fail, false, nil
	}
	// never time out a paused deployment
	if d.Status == structs.DeploymentStatusPaused {
	return false, false, nil
	}
	fail = false

[tehut]

Works for me! Swapped it to false.

But now you have me wondering if maybe it was reset to false here just to make sure the value was zero'd out before the stanza below where it could be set to true. Which really shouldn't be an issue but things get weird in this long running loops so 🤷

[gulducat]

your patch isn't quite the same as my suggestion, but it was a nit anyway 🙃

[github-actions]

I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.