Fix API response handling for staging resources #3

Open
graham wants to merge 2 commits from fix/api-response-envelope into main
Owner

Decode site, device, and check API response envelopes and preserve the redacted organization SNMP community. Verified with go test, go vet, and a staging apply followed by a no-change plan.

Decode site, device, and check API response envelopes and preserve the redacted organization SNMP community. Verified with go test, go vet, and a staging apply followed by a no-change plan.
Fix API response handling for staging resources
Some checks failed
skippy-bot/review Skippy review: 3 blocking findings open — see the PR thread
Release / goreleaser (push) Failing after 30s
skippy-bot/autofix Repaired by the auto-fix in 6774d081
e414395e3c
skippy-bot left a comment

🤖 Skippy PR review

3 findings — 3 blocking before merge.

Severity Location Issue
🟠 High internal/provider/site_resource_test.go:377 Site SNMP community is wiped on create because the fake API invents a field the real one redacts
🟠 High internal/provider/client.go:235 Decoding the device envelope exposes the same wiped write-only secret: SNMPv3 passwords
🟡 Warning internal/provider/client.go:159 Six of ten registered resources still decode the envelope into a bare struct

Reviewed e414395e3c3f. Comment skippy review to re-run.

### 🤖 Skippy PR review **3 findings** — 3 blocking before merge. | Severity | Location | Issue | | --- | --- | --- | | 🟠 High | `internal/provider/site_resource_test.go:377` | Site SNMP community is wiped on create because the fake API invents a field the real one redacts | | 🟠 High | `internal/provider/client.go:235` | Decoding the device envelope exposes the same wiped write-only secret: SNMPv3 passwords | | 🟡 Warning | `internal/provider/client.go:159` | Six of ten registered resources still decode the envelope into a bare struct | <sub>Reviewed `e414395e3c3f`. Comment `skippy review` to re-run.</sub> <!-- skippy-pr-review -->
@ -155,3 +157,3 @@
}
return &result, nil
return &result.Data, nil
First-time contributor

🟡 Warning — Six of ten registered resources still decode the envelope into a bare struct

This PR converts the site, device and check clients to a wrapper struct with a data JSON tag, but the contract holds for every API v1 endpoint (ToweropsWeb.Api.V1.ResourceController moduledoc: every JSON success is a data envelope) and 18 functions still unmarshal into a bare struct: schedules (:279, :294, :310), escalation policies (:341, :356, :372), agents (:409, :424), integrations (:464, :479, :495), maintenance windows (:530, :545, :561), coverages (:736, :751, :769, :791). Each decodes to zero values, so create stores an empty id and the following refresh or destroy requests the collection path with no id. All six resources are registered in provider.go.

This is a scope call rather than a repair, so it is not auto-fixed: either extend this PR with the same wrapper or split a follow-up before calling response handling fixed.

**🟡 Warning — Six of ten registered resources still decode the envelope into a bare struct** This PR converts the site, device and check clients to a wrapper struct with a `data` JSON tag, but the contract holds for every API v1 endpoint (`ToweropsWeb.Api.V1.ResourceController` moduledoc: every JSON success is a `data` envelope) and 18 functions still unmarshal into a bare struct: schedules (`:279`, `:294`, `:310`), escalation policies (`:341`, `:356`, `:372`), agents (`:409`, `:424`), integrations (`:464`, `:479`, `:495`), maintenance windows (`:530`, `:545`, `:561`), coverages (`:736`, `:751`, `:769`, `:791`). Each decodes to zero values, so create stores an empty `id` and the following refresh or destroy requests the collection path with no id. All six resources are registered in `provider.go`. This is a scope call rather than a repair, so it is not auto-fixed: either extend this PR with the same wrapper or split a follow-up before calling response handling fixed.
@ -223,3 +233,3 @@
}
return &result, nil
return &result.Data, nil
First-time contributor

🟠 High — Decoding the device envelope exposes the same wiped write-only secret: SNMPv3 passwords

Now that GetDevice/CreateDevice/UpdateDevice decode the payload, the read path really runs - and DeviceSerializer returns only snmpv3_auth_password_set / snmpv3_priv_password_set, never the passwords. device_resource.go:402 and :414 (Read) assign types.StringNull() when the response omits them, and :626/:638 (Update) do the same, so a device configured with snmpv3_auth_password or snmpv3_priv_password loses both from state on the first refresh and the next update apply fails with "Provider produced inconsistent result after apply" (both attributes are Optional, Sensitive, not Computed). No test sets either password, which is why the suite is green.

Same fix as the organization: leave the planned value in data when the API omits the field.

**🟠 High — Decoding the device envelope exposes the same wiped write-only secret: SNMPv3 passwords** Now that `GetDevice`/`CreateDevice`/`UpdateDevice` decode the payload, the read path really runs - and `DeviceSerializer` returns only `snmpv3_auth_password_set` / `snmpv3_priv_password_set`, never the passwords. `device_resource.go:402` and `:414` (Read) assign `types.StringNull()` when the response omits them, and `:626`/`:638` (Update) do the same, so a device configured with `snmpv3_auth_password` or `snmpv3_priv_password` loses both from state on the first refresh and the next update apply fails with "Provider produced inconsistent result after apply" (both attributes are Optional, Sensitive, not Computed). No test sets either password, which is why the suite is green. Same fix as the organization: leave the planned value in `data` when the API omits the field.
skippy-bot marked this conversation as resolved
@ -402,3 +376,2 @@
w.WriteHeader(http.StatusCreated)
json.NewEncoder(w).Encode(Site{
ID: siteID,
json.NewEncoder(w).Encode(map[string]Site{"data": {ID: siteID,
First-time contributor

🟠 High — Site SNMP community is wiped on create because the fake API invents a field the real one redacts

This test only passes because its handler returns snmp_community, which the API never sends: ToweropsWeb.Api.SiteSerializer exposes snmp_community_set and no credential value. Against the real API created.SNMPCommunity is nil, setSiteOptionalFields (site_resource.go:270) assigns types.StringNull(), and since snmp_community is Optional and not Computed, Terraform aborts the create with "Provider produced inconsistent result after apply"; on refresh the value is dropped from state and every later plan re-proposes it.

This PR already gives towerops_organization the right treatment (keep the planned value, never overwrite it from the response). Do the same here: drop the else { types.StringNull() } branch for SNMPCommunity in setSiteOptionalFields, and make this fake return snmp_community_set instead of the secret, the way TestAccOrganizationResource_redactedCommunity does.

**🟠 High — Site SNMP community is wiped on create because the fake API invents a field the real one redacts** This test only passes because its handler returns `snmp_community`, which the API never sends: `ToweropsWeb.Api.SiteSerializer` exposes `snmp_community_set` and no credential value. Against the real API `created.SNMPCommunity` is nil, `setSiteOptionalFields` (`site_resource.go:270`) assigns `types.StringNull()`, and since `snmp_community` is Optional and not Computed, Terraform aborts the create with "Provider produced inconsistent result after apply"; on refresh the value is dropped from state and every later plan re-proposes it. This PR already gives `towerops_organization` the right treatment (keep the planned value, never overwrite it from the response). Do the same here: drop the `else { types.StringNull() }` branch for `SNMPCommunity` in `setSiteOptionalFields`, and make this fake return `snmp_community_set` instead of the secret, the way `TestAccOrganizationResource_redactedCommunity` does.
skippy-bot marked this conversation as resolved
fix(review): site SNMP community is wiped on create because the fake API
Some checks failed
skippy-bot/autofix Auto-fix in 6774d081: 1 finding(s) resolved, 3 file(s) changed. CI is the referee.
skippy-bot/review Skippy review: 1 blocking finding open — see the PR thread
6774d08150
Skippy-Autofix: internal/provider/site_resource_test.go:377 Site SNMP community is wiped on create because the fake API invents a field the real one redacts
Skippy-Autofix: internal/provider/client.go:235 Decoding the device envelope exposes the same wiped write-only secret: SNMPv3 passwords
First-time contributor

Auto-fix pushed by skippy-bot in 6774d081.

  • internal/provider/site_resource.go:270 dropped the else { data.SNMPCommunity = types.StringNull() } branch in setSiteOptionalFields so the planned snmp_community is kept when the API redacts it
  • internal/provider/site_resource_test.go:379 fake POST/GET handlers now return snmp_community_set: true (via map[string]any) instead of echoing the secret the real API never sends
  • internal/provider/device_resource.go:405 removed the null-out branch for SNMPv3AuthPassword in Read
  • internal/provider/device_resource.go:417 removed the null-out branch for SNMPv3PrivPassword in Read
  • internal/provider/device_resource.go:629 removed the null-out branch for SNMPv3AuthPassword in Update
  • internal/provider/device_resource.go:641 removed the null-out branch for SNMPv3PrivPassword in Update

Still open, not mechanical: none.
Not re-reviewed: this repair came from the reviewer's own findings, so the fix pass closes what it fixed and CI is the referee.

Auto-fix pushed by skippy-bot in `6774d081`. - internal/provider/site_resource.go:270 dropped the `else { data.SNMPCommunity = types.StringNull() }` branch in `setSiteOptionalFields` so the planned `snmp_community` is kept when the API redacts it - internal/provider/site_resource_test.go:379 fake POST/GET handlers now return `snmp_community_set: true` (via `map[string]any`) instead of echoing the secret the real API never sends - internal/provider/device_resource.go:405 removed the null-out branch for `SNMPv3AuthPassword` in `Read` - internal/provider/device_resource.go:417 removed the null-out branch for `SNMPv3PrivPassword` in `Read` - internal/provider/device_resource.go:629 removed the null-out branch for `SNMPv3AuthPassword` in `Update` - internal/provider/device_resource.go:641 removed the null-out branch for `SNMPv3PrivPassword` in `Update` Still open, not mechanical: none. Not re-reviewed: this repair came from the reviewer's own findings, so the fix pass closes what it fixed and CI is the referee. <!-- skippy-pr-review -->
First-time contributor

Correction to the auto-fix note above (it said "Still open, not mechanical: none"):

  • The device finding (comment 30033) is repaired in 6774d081 and resolved there, so it is closed, not open.
  • The envelope-scope warning (comment 30034) is still open on purpose: only sites, devices, checks and the organization decode the data envelope; schedules, escalation policies, agents, integrations, maintenance windows and coverages still read bare structs. Your call whether that belongs in this PR or a follow-up.

Two things worth knowing before you merge: this repo's only workflow is Release on tags, so no CI grades the push; and helper/resource.Test skips unless TF_ACC is set (no test case sets IsUnitTest), so go test ./internal/provider alone skips every TestAcc* fake, including the new organization redaction test. TF_ACC=1 go test ./internal/provider is the run that exercises them.

Correction to the auto-fix note above (it said "Still open, not mechanical: none"): - The device finding (comment 30033) is repaired in `6774d081` and resolved there, so it is closed, not open. - The envelope-scope warning (comment 30034) **is still open on purpose**: only sites, devices, checks and the organization decode the `data` envelope; schedules, escalation policies, agents, integrations, maintenance windows and coverages still read bare structs. Your call whether that belongs in this PR or a follow-up. Two things worth knowing before you merge: this repo's only workflow is Release on tags, so no CI grades the push; and `helper/resource.Test` skips unless `TF_ACC` is set (no test case sets `IsUnitTest`), so `go test ./internal/provider` alone skips every `TestAcc*` fake, including the new organization redaction test. `TF_ACC=1 go test ./internal/provider` is the run that exercises them. <!-- skippy-pr-review -->
Some checks failed
skippy-bot/autofix Auto-fix in 6774d081: 1 finding(s) resolved, 3 file(s) changed. CI is the referee.
skippy-bot/review Skippy review: 1 blocking finding open — see the PR thread
This pull request can be merged automatically.
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin fix/api-response-envelope:fix/api-response-envelope
git switch fix/api-response-envelope

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch main
git merge --no-ff fix/api-response-envelope
git switch fix/api-response-envelope
git rebase main
git switch main
git merge --ff-only fix/api-response-envelope
git switch fix/api-response-envelope
git rebase main
git switch main
git merge --no-ff fix/api-response-envelope
git switch main
git merge --squash fix/api-response-envelope
git switch main
git merge --ff-only fix/api-response-envelope
git switch main
git merge fix/api-response-envelope
git push origin main
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
graham/terraform-provider-towerops!3
No description provided.