Subnetting Strategy

Cloud

AWS, Azure, and GCP subnetting — the constraints that matter, the patterns that work, and the mistakes that cause outages.

The #1 cloud subnetting lesson

IP space is the long pole in the tent. Compute scales in seconds (auto-scaling groups, spot fleets, EKS node pools). Storage scales instantly (EBS, PD). But you cannot resize a subnet in place on AWS or Azure. If your /24 runs out of IPs during a traffic spike, you face a painful migration — not a button click. IP planning is the one thing that must be right before you need it.

Case study: the Neon outage (May 2025). Pods grew from ~5,000 to ~8,100, exhausting IPs across three /20 subnets. 5.5 hours of downtime. The failure was subtle — AWS VPC CNI's warm ENI buffer stranded IPs on nodes that had no CPU capacity. Even though the subnet showed "available" IPs, prefix allocations within ENIs were exhausted. https://neon.com/blog/aws-cni-lessons-from-a-production-outage

GCP is the exception

GCP supports expanding subnet ranges in place (increase only, never shrink). AWS and Azure do not — you must create a new subnet and migrate. https://cloud.google.com/vpc/docs/using-vpc#expand-subnet

Reserved addresses by provider

AWS reserves .0 (network), .1 (VPC router), .2 (DNS), .3 (future use), and the last address (broadcast). https://docs.aws.amazon.com/vpc/latest/userguide/subnet-sizing.html · https://learn.microsoft.com/en-us/azure/virtual-network/virtual-networks-faq · https://cloud.google.com/vpc/docs/subnets

AWS 3-tier VPC pattern

/16 VPC with /24 subnets across 3 AZs. Always start with /16 — you can add secondary CIDRs later but cannot shrink. https://docs.aws.amazon.com/vpc/latest/userguide/configure-subnets.html

AWS EKS pod networking (2026)

Prefix delegation is now the standard approach. VPC CNI assigns /28 blocks (16 IPs) to ENI slots instead of individual IPs, enabling 110 pods/node. Use /20 or larger subnets for pods. Karpenter sets --max-pods=110 by default, ignoring the legacy ENI calculator.

For pod CIDRs that won't conflict with corporate space, add a secondary CIDR from 100.64.0.0/10 (CGNAT / RFC 6598). https://docs.aws.amazon.com/eks/latest/best-practices/vpc-cni.html · https://www.eksworkshop.com/docs/networking/vpc-cni/prefix/

Azure hub-and-spoke

Hub VNet /16 for shared services + firewall. Spoke VNets per environment. Azure CNI Overlay is the 2026 recommendation for AKS: pods get addresses from an overlay CIDR (default 10.244.0.0/16) that can be reused across independent clusters in the same VNet. https://learn.microsoft.com/en-us/azure/aks/azure-cni-overlay · https://learn.microsoft.com/en-us/azure/virtual-network/concepts-and-best-practices

GCP global VPC

Custom mode, regional subnets. GKE now supports Class E addresses (240.0.0.0/4) for pods — 268M addresses, dwarfing all of RFC 1918 combined. Production-ready but with caveats: Class E traffic gets blocked on Windows hosts and some on-prem hardware. https://cloud.google.com/blog/products/containers-kubernetes/how-class-e-addresses-solve-for-ip-address-exhaustion-in-gke

Multi-region IP planning

Assign each region a /12 from a 10.0.0.0/8 enterprise allocation:

10.0.0.0/8 (enterprise)
  10.0.0.0/12  = us-east-1
  10.16.0.0/12 = us-west-2
  10.32.0.0/12 = eu-west-1
  10.48.0.0/12 = ap-southeast-1
  10.64.0.0/12 = future growth

https://docs.aws.amazon.com/wellarchitected/latest/reliability-pillar/rel_planning_network_topology_non_overlap_ip.html

Sizing

Right-sizing subnets — the math, the mistakes, and the tradeoffs. Getting this wrong costs either wasted address space or a painful renumbering project.

Kubernetes CIDR sizing

The formula: pod CIDR = (max nodes) × (IPs/node).

Rolling updates temporarily double IP usage (old + new pods coexist). Always size for 2x peak pod count. https://docs.aws.amazon.com/eks/latest/best-practices/subnets.html · https://learn.microsoft.com/en-us/azure/aks/concepts-network-ip-address-planning · https://cloud.google.com/kubernetes-engine/docs/how-to/flexible-pod-cidr

Cilium and eBPF

Cilium is now the dominant advanced CNI (used by Adobe, Capital One; Azure AKS moving toward it). It uses identity-based policies (Kubernetes labels) instead of IP-based rules — a paradigm shift. But pods still need IPs for the data plane, so CIDR planning remains necessary. https://github.com/cilium/cilium

Service mesh impact

Istio/Linkerd sidecars share the pod's network namespace, so they don't consume extra IPs. The main impact is on pod density: sidecar memory/CPU overhead reduces how many pods fit per node, which indirectly affects how many node IPs you need.

Multi-cluster

All clusters in a mesh must have non-overlapping pod CIDRs. Cilium ClusterMesh requires a native routing CIDR covering all clusters (typically 10.0.0.0/8). Plan pod CIDRs per cluster from day one. https://docs.cilium.io/en/stable/network/clustermesh/clustermesh/

The 2x rule

Size for 2x your current device count. This accounts for 3–5 years of organic growth without renumbering. It's the single most reliable heuristic in subnet planning.

Over-provisioning vs under-provisioning

Sizing by workload type

The power-of-two trap

For newcomers: 100 devices needs a /25 (126 usable) or a /24 (254 usable). You can't get exactly 100. A /25 is cutting it close — go /24 for the 2x growth rule.

Efficiency vs sprawl

In a /8 you have 16M addresses. Wasting a few /24s is negligible. But in cloud where each VPC is a /16, every /24 you allocate is 1/256th of your VPC. Sprawl becomes real when you have 50 microservices each demanding their own subnet. The fix: use fewer, larger subnets and enforce isolation at the security group / network policy layer.

Branch

Branch office IP planning — standardized templates, WAN links, and summarization-friendly allocation.

Standardized branch template

A /22 per branch gives 1,024 addresses. Use the same VLAN layout at every branch — consistency makes troubleshooting and automation possible.

Branch 2 = 10.20.4.0/22, Branch 3 = 10.20.8.0/22. Each branch summarizes to one route. Note the intentional gap: increment by 4 in the third octet to stay on /22 boundaries. The growth reserve /25 within each branch provides room for expansion.

Hub-and-spoke WAN

Use /31 per point-to-point link (RFC 3021). Reserve 10.255.0.0/24 for all WAN links — that's 128 links. https://datatracker.ietf.org/doc/html/rfc3021

SD-WAN

The overlay simplifies routing policy but underlay IPs still matter. Each hub needs at least a /24; spokes use the standard branch template above.

Summarization-friendly allocation

This is a critical concept. Bad: Branch 1 = 10.20.1.0/24, Branch 2 = 10.20.5.0/24 (can't summarize). Good: Branch 1 = 10.20.0.0/22, Branch 2 = 10.20.4.0/22 (all 8 branches summarize to 10.20.0.0/19). Always allocate on power-of-two boundaries.

DHCP scope planning

Reserve the first 10–20 IPs for static assignments (gateway, printers, servers). Recommended lease times: 8 hours for user devices, 4 hours for guests, infinite for VoIP phones. Use DHCP Option 150 for phone provisioning server addresses.

RFC 1918 tradeoffs

Recommendation: use 10/8 as primary. Reserve 172.16/12 for DMZ or environments that must not overlap with 10.x. Use 192.168/16 only for OOB management. https://datatracker.ietf.org/doc/html/rfc1918

Data Center

Data center fabric subnetting — spine-leaf underlay, VXLAN overlay, out-of-band management, and multi-tenant design.

Spine-leaf underlay

/32 loopbacks for each device, /31 point-to-point links between spines and leaves. Convention: 3rd octet = tier (0 = spine, 1 = leaf), 4th octet = device number.

Loopbacks (10.0.0.0/24):
  Spines: 10.0.0.1/32, 10.0.0.2/32, 10.0.0.3/32, 10.0.0.4/32
  Leaves:  10.0.1.1/32, 10.0.1.2/32, ... 10.0.1.48/32

P2P links (10.0.2.0/24):
  Spine1-Leaf1: 10.0.2.0/31
  Spine1-Leaf2: 10.0.2.2/31
  Spine2-Leaf1: 10.0.2.4/31

eBGP underlay, one ASN per device or per tier. ECMP across all spines. https://www.juniper.net/documentation/us/en/software/nce/sg-005-data-center-fabric/

OOB management

A physically separate network (dedicated switches, separate cabling). Use 192.168.x.x to clearly distinguish from production 10.x.x.x. Access via a dedicated jump host only. https://www.cisco.com/c/en/us/solutions/collateral/service-provider/out-of-band-best-practices-wp.html

East-west vs north-south

VXLAN overlay for east-west traffic (server-to-server within the DC). Border leaf for north-south traffic (DC to the outside world). Every east-west path in a spine-leaf fabric is exactly 2 hops (leaf→spine→leaf).

Multi-tenant (VXLAN-EVPN)

VRF per tenant, VNI per segment. Overlapping IPs are allowed across tenants because VRFs provide full routing isolation. https://www.cisco.com/c/en/us/products/collateral/switches/nexus-9000-series-switches/white-paper-c11-739942.html

Migration and renumbering

When renumbering is unavoidable, four strategies minimize disruption:

• Parallel prefix — add the new range alongside the old on every interface; migrate traffic gradually
• DHCP-first — handles 80–90% of devices automatically at next lease renewal
• DNS-based — services accessed by hostname migrate transparently as DNS records are updated
• Phased cutover — one rack at a time, maintaining rollback capability throughout

IPv6 in the data center

A /48 per site gives 65,536 /64 subnets. Use /127 for point-to-point links (RFC 6164) instead of /64 to prevent ping-pong attacks. Align to nibble boundaries (prefix lengths divisible by 4) for clean DNS reverse zones. https://www.ripe.net/publications/docs/ripe-690/

IPAM tools

You need something better than a spreadsheet. Options:

• NetBox — gold standard, open source, full IPAM + DCIM
• phpIPAM — lightweight, open source, web-based
• AWS VPC IPAM — native to AWS, tracks VPC allocations
• Infoblox / BlueCat — enterprise-grade, DDI (DNS/DHCP/IPAM)

Documentation minimum per subnet: CIDR, VLAN, purpose, site, gateway, DHCP range, date allocated. Audit quarterly.