KVM: SystemVMs lose network after migration when a shared Network with security group and the public network share a VLAN #14518

problem

In an advanced zone with security groups, if a shared guest network uses the same VLAN on the same physical device as the public network, both resolve to the same Linux bridge (breth1-51 here). The first security-grouped user VM on a host installs the bridge firewall framework on that bridge, including two catch-all rules:

-A FORWARD -o breth1-51 -j DROP
-A FORWARD -i breth1-51 -j DROP

The SSVM and CPVM have their public NICs on that same bridge but are never programmed into the framework, so every new inbound flow to them is dropped. Established connections survive only while they stay in conntrack via the RELATED,ESTABLISHED rule in BF-breth1-51, which is why it looks like it works for a while and then breaks.

Packet path for a new inbound connection to the system VM:

FORWARD -> BF-breth1-51 -> not RELATED,ESTABLISHED
        -> BF-breth1-51-OUT (empty, no match)
        -> --physdev-out <uplink> ACCEPT (no match, physdev-out is vnetX)
        -> falls through to "-A FORWARD -o breth1-51 -j DROP"

The missing rule is -A BF-breth1-51-OUT -m physdev --physdev-is-bridged --physdev-out -j s--VM.

Three gaps in the code combine to cause this:

1. System VMs are never programmed in an advanced zone. LibvirtComputingResource.applyDefaultNetworkRules() only calls configureDefaultNetworkRulesForSystemVm() when a NIC reports isSecurityGroupEnabled(), and NetworkModelImpl.isSecurityGroupSupportedInNetwork() returns false for anything that is not TrafficType.Guest. SSVM and CPVM have only control, management and public NICs.
2. Migration does not re-apply them. SecurityGroupManagerImpl.handleVmMigrated() returns before reaching the system VM branch:

protected void handleVmMigrated(VMInstanceVO vm) {
    if (!isVmSecurityGroupEnabled(vm.getId())) {
        return;                                    // always true for system VMs
    }
    if (vm.getType() != VirtualMachine.Type.User) {
        ... NetworkRulesSystemVmCommand ...        // unreachable

isVmSecurityGroupEnabled() inspects the VM’s own NICs, which is only meaningful for user VMs. NetworkRulesSystemVmCommand has exactly one sender in the codebase — this line — so the command is effectively dead code today.

3. Nothing re-programs existing VMs when the framework appears. add_fw_framework() in scripts/vm/network/security_group.py installs the chains and the two catch-all DROP rules but does not program the system VMs already attached to that bridge. The framework is also never torn down — nothing removes the FORWARD DROP rules or the BF- chains once created.

By the same mechanism, any virtual router with a public NIC on that bridge is expected to be affected; this has not been separately verified.

versions

4.22.1.1. Reproduced in a lab and observed at a production site.

The steps to reproduce the bug

1. Advanced zone, KVM, security groups enabled.
2. Create a shared guest network on the same VLAN and physical device as the public network, so both use the same bridge (breth1-51).
3. Start a user VM on the shared network on host A. This triggers add_fw_framework() on breth1-51.
4. Migrate the SSVM or CPVM onto host A.

Expected: the system VM stays reachable, with its vifs programmed into BF-breth1-51-IN / BF-breth1-51-OUT on the destination host.

Actual: the system VM loses connectivity for new inbound flows. On the destination host:

# iptables -S FORWARD | grep breth1-51
-A FORWARD -o breth1-51 -m physdev --physdev-is-bridged -j BF-breth1-51
-A FORWARD -i breth1-51 -m physdev --physdev-is-bridged -j BF-breth1-51
-A FORWARD -o breth1-51 -j DROP
-A FORWARD -i breth1-51 -j DROP

# iptables -S BF-breth1-51-OUT
-N BF-breth1-51-OUT          <-- empty, no per-vif jumps

What to do about it?

Workaround: remove the catch-all DROP rules on the affected bridge. They are not re-created unless the BF- chains themselves are removed, so this holds until the host reboots or iptables is flushed:

iptables -D FORWARD -o breth1-51 -j DROP
iptables -D FORWARD -i breth1-51 -j DROP

Security group enforcement for user VMs is unaffected: each programmed vif still default-denies in its own chain (i---VM ends with -j DROP, and the egress chain ends with -j DROP when egress rules are defined). The catch-all only governs vifs that have no per-VM chain at all.

Fix is hree changes, none of which creates a bridge firewall framework where one does not already exist:

1. In SecurityGroupManagerImpl.handleVmMigrated(), move the isVmSecurityGroupEnabled() guard into the user VM branch so system VMs reach NetworkRulesSystemVmCommand.
2. Give configureDefaultNetworkRulesForSystemVm() an “only existing framework” mode, passed by LibvirtNetworkRulesSystemVmCommandWrapper (the migration path) and by a new fallback in applyDefaultNetworkRules() for system VMs with no security-group-enabled NIC. In security_group.py, default_network_rules_systemvm() then filters out bridges with no framework rather than creating one.
3. In add_fw_framework(), program the system VMs already attached to the bridge at the point the framework is created, so a system VM that arrives before the first user VM is also covered.