Skip to content

Quota charges attached volumes twice since 4.22.1 (cumulative + per-VM usage_volume records from #11531) #14550

Open

Description

@MacroHarry

problem

Since #11531 (4.22.1), Usage intentionally writes two kinds of usage_volume helper rows for an attached volume: a cumulative row (vm_id NULL) and a per-VM row (vm_id set). Both generate an hourly usage_type = 6 (VOLUME) record in cloud_usage, one with and one without vm_instance_id.

The Quota plugin treats both records as billable, so every attached volume is charged twice.

versions

  • CloudStack 4.22.1.1 (management, usage)
  • KVM, Ceph RBD primary storage
  • Quota enabled with a volume tariff (usage_type = 6)

steps to reproduce

  1. Enable Quota and configure a tariff for usage type 6.
  2. Create a data volume and attach it to a running VM (events VOLUME.CREATE, then VOLUME.ATTACH).
  3. After the next Usage run, check cloud_usage.usage_volume. There are two rows with deleted IS NULL, one with vm_id NULL and one with vm_id set to the VM.
  4. Check cloud_usage.cloud_usage for that volume and hour. There are two usage_type = 6 records with the same raw_usage: one with vm_instance_id NULL and one with vm_instance_id set.
  5. Check cloud_usage.quota_usage. There is one row per cloud_usage record, both with the same quota_used, so the account balance is debited twice.

Example, one volume and one hour:

cloud_usage.vm_instance_id records quota_used
NULL (cumulative) 1 0.0081
set (per-VM) 1 0.0081

expected

A volume should be charged once per hour. Quota should bill only one of the two record kinds, for example the cumulative record (vm_instance_id NULL), since that is the one that is consistent with pre-#11531 behaviour.

notes

  • Today, root volumes on new VMs don't show the double charge, because the second usage_volume insert fails on the old (volume_id, created) unique key (usage: widen usage_volume unique key to include vm_id #13909 / backport: usage: widen usage_volume unique key to include vm_id (#13909) #13933). Once that key is widened in 4.22.2, root volumes will get both rows too, and will also be double-charged by Quota.
  • Our workaround: an hourly job sets deleted = created on per-VM usage_volume rows whenever an active cumulative row exists for the same volume. Per-VM records then stop being generated for that volume, and the cumulative row keeps billing it through detach and re-attach. Records written before we found this had to be removed by hand.

Activity

  1. boring-cyborg commented on Nov 1, 2026

    @boring-cyborg

    Thanks for opening your first issue here! Be sure to follow the issue template!

  2. github-actions commented on Nov 2, 2026

    @github-actions

    Warning

    threat detection engine error
    The threat detection engine encountered an error and could not complete analysis. This is a tooling failure, not a security finding.

    What happened

    The threat detection engine failed to produce results.

    Review the workflow run logs for details.

    🎯 Triage report

    Since #11531 (4.22.1), Usage writes two usage_volume helper rows for an attached volume: a cumulative row (vm_id NULL) and a per-VM row. Each generates an hourly usage_type = 6 record, and Quota bills both, so attached volumes are charged twice. The reporter describes a workaround that sets deleted on per-VM rows, and notes that the double charge will also reach root volumes once the unique key is widened in 4.22.2 (#13909 / #13933).

    📊 Assessment

    Dimension Value Reasoning
    Type type:bug Duplicate billing records since 4.22.1
    Component component:usage-server, component:quota Usage writes the rows; Quota debits them
    Severity Severity:Major Account balances are debited twice for attached volumes; workaround in place
    Labels type:bug, component:usage-server, component:quota, Severity:Major
    Coding agent Not suitable Requires a decision on which record kind is billable and how existing bad records are corrected

    🔗 Similar issues

    💡 Notes and suggestions
    • Check the 4.22.2 plan for the unique-key change before it ships, since it would extend the double charge to root volumes.
    • Existing duplicate cloud_usage and quota_usage rows will need a cleanup or credit decision; the reporter removed theirs by hand.
    • Decide whether to bill only the cumulative record, as the reporter proposes, and check that against Track volume usage data at a vm granularity as well #11531's intent.

    Generated by Daily Issue Triage · copilot · haiku55 · ~2 AIC · ⊞ 3.8K · ◷

    Add this agentic workflow to your repo

    To install this agentic workflow, run

    gh aw add githubnext/agentics/workflows/daily-issue-triage.md@139fa09ef17cde4135e696ff8e7af09bc5ddfa6e
    
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions