[multicast] implicit group lifecycle with IP pool integration

[zeeshanlakhani]

This PR also addresses permission models, object deletion, and error handling questions related to reserved addresses presented in @askfongjojo's testing Google Doc (default IP Pools are covered in a follow-up, stacked PR).

In thinking through the Groups API, permission scopes, and flexibility, @rcgoodfellow mentioned this consideration:

Do we need an explicit notion of a group object at all? Or can
instances simply allocate/deallocate group IPs from pools, and there is
no explicit management of group objects.

With Fleet admins having access control to create pools and link silos to a pool, we arrived at the idea of replacing the current explicit multicast group CRUD with an implicit lifecycle, where groups are created upon the first member join and deleted when the last member leaves.

Note: Most of the PR's changes are test-related due to moving away from the explicit multicast group(s) lifecycle.

Auth Model:

• Discovery (fleet-scoped):
  • Read/list groups and list members: any authenticated user in the same fleet.
• Membership (project-scoped):
  • Join/leave requires Instance::Modify on the specific instance.
• Creation control:
  • Implicit group creation only when the caller’s silo is linked to a suitable multicast pool (by name or by explicit IP in that pool).

Behavior:

• Implicit lifecycle:
  • Create on first join (idempotent); delete when last member leaves (atomic mark-for-removal, reconciler schedules cleanup).
• Addressing and validation:
  • Implicit allocation from the caller’s linked multicast pools.
  • SSM/ASM semantics enforced:
    • IPv4 SSM 232/8 and IPv6 ff3x::/32 require ≥1 source IP.
    • ASM must not specify sources. - When joining by explicit IP: resolve the pool containing the IP, and we verify the silo link before creating.
• Error handling: - Reserved/invalid multicast ranges rejected at pool/range add time.

API:

• Primary flows: - Group-centric member management: POST/DELETE /v1/multicast-groups/{group}/members - Instance-centric join/leave: PUT/DELETE /v1/instances/{instance}/multicast-groups/{group}
• Discovery endpoints remain for list/view; there is no explicit group create/update/delete.
• This is a breaking change, but multicast is not yet enabled or available in production

Key changes:

• Implicit group model; groups exist while they have members.
• IP pool integration for multicast allocation with silo link gating.
• Simplified API centered on join/leave flows.
• Add multicast_ip to the member table for responses.
• For consistency, move to Instant type over SystemTime for mcast-related caches

Follow‑ups (stacked [and related] PRs)

• omdb additions for multicast introspection -> [multicast] add omdb mcast commands for groups, members, pools #9464
• Remove MVLAN from group data model -> [multicast] Drop (m)vlan from multicast groups #9451
• Default IP pool support (IPv4/IPv6 × unicast/multicast) -> [ip-pools] Allow multiple default IP pools per silo #9452 (stacked atop [multicast] Drop (m)vlan from multicast groups #9451 due to database migration conflict, but can be reviewed separately)
• Dendrite: use omicron-common constants for validation -> [multicast] Narrow admin-scoped to admin-local only (ff04::/16) dendrite#162

[askfongjojo]

@zeeshanlakhani - The revised authn model and implicit creation/deletion logic make sense to me. I'll make another pass tomorrow to ensure I didn't miss anything but the mental model of multicast group lifecycle seems straightforward enough.

[askfongjojo]

The validation against ASM/SSM boundary spanning is not part of this PR but reading this comment causes me to wonder how to use IP pools with ASM vs SSM IP ranges. Let's say a silo has two distinct types of workloads that have different source IP filtering requirements, how would the operator configure the silo's default IP pool such that users have the control over getting an ASM vs SSM address for a new multicast group (since they can't specify the IP pool to use in a join request)?

[zeeshanlakhani]

We handle this in allocate_external_multicast_group:

// Determine if this is an SSM request (source_ips provided) or an
// implicit ASM request (no sources, no explicit pool/IP)
let sources_empty = params.source_ips.as_ref().map(|v| v.is_empty()).unwrap_or(true);
let needs_ssm_pool = !sources_empty && params.pool.is_none() && params.ip.is_none();
let needs_asm_pool = sources_empty && params.pool.is_none() && params.ip.is_none();

// Select the appropriate pool:
// - If `source_ips` provided without explicit pool/IP, find an SSM pool.
// - If no `source_ips` and no explicit pool/IP, find an ASM pool.
let authz_pool = if needs_ssm_pool {
    let (authz_pool, _) = self
        .ip_pools_fetch_ssm_multicast(opctx)
        .await
        ...
} else if needs_asm_pool {
    let (authz_pool, _) = self
        .ip_pools_fetch_asm_multicast(opctx)
        .await
        ...

If the user provides source_ips in the join request body, we'll select an SSM pool automatically; if a user doesn't provide source_ips, we'll select an ASM pool
automatically.

The operator setup involves creating a multicast pool with SSM ranges (232.x.x.x for IPv4, ff3x::/32 for IPv6) and/or another multicast pool with ASM ranges (other
multicast addresses), then linking both to the silo. One can be marked as the default, but the SSM/ASM selection criteria scans all linked pools by their IP ranges.

It's really all about source IPs (or not).

[rcgoodfellow]

Thanks @zeeshanlakhani for getting the implicit group changes coded up. Questions/comments follow.

[zeeshanlakhani]

I'll update the schema/versions next review @rcgoodfellow @internet-diglett.