Data Center Networking: Architecture and Equipment Selection

Modern data center architecture (2026) is built on spine-leaf EVPN-VXLAN, 100G/400G links between spine switches, bonded 25G server connections, and intent-based automation. Legacy 3-tier Cisco Nexus 7000 architectures are being phased out. This guide covers architecture, sizing, cooling, power, connectivity, and 2026 AI/ML data center trends.

Modern Data Center Design Principles

• Spine-leaf is mandatory (no 3-tier architecture)
• No stretched L2 networks (L3 routed access + VXLAN overlay)
• Multi-tenancy through VRF/EVPN tenants
• Automation: Ansible, Apstra, ACI, NSX-T
• Observability: streaming telemetry, Prometheus, Grafana
• Security: hardware-based microsegmentation (Pensando, NSX-T)

Spine-Leaf Topology

Small (1 fabric)

• 2-4 spine switches with 32×100G ports (e.g., Nexus 9336C-FX2)
• 8-32 leaf switches with 48×25G ports + 6-8×100G uplinks (Nexus 93180YC-FX)
• Typical oversubscription ratio: 3:1
• Capacity: 200-1500 servers with 25G connectivity

Large (super-spine, multi-pod)

• Super-spine: 8× Nexus 9336C-GX (36×400G)
• Spine switches per pod: 4× Nexus 9336C-FX2
• Leaf switches per pod: 16-32× Nexus 93180YC-FX
• Pods: 4-8 per data center
• Capacity: 5000-30 000 servers

Server Connectivity

• Enterprise 1U servers: 2× 10G SFP+ MC-LAG (HP DL380, Dell R750)
• AI/ML GPU servers (A100, H100): 2-4× 100G InfiniBand or RoCE Ethernet
• Hyperconverged infrastructure (Nutanix, VxRail): 2× 25G SFP28
• SAN storage: 32G Fibre Channel (declining) or 25G iSCSI
• NVMe-oF over Ethernet: 100G RoCEv2

Data Center Storage

• All-flash arrays (Pure Storage, NetApp AFF, Dell PowerStore): 1-20 PB
• NVMe-oF (NVMe over Fabrics): 25G/100G
• Object storage (Ceph, MinIO, S3-compatible): for backup and archiving
• Backup: Veeam + Cohesity + LTO-9 tape (18 TB native)

Cooling and Power

• Target PUE: 1.3-1.5 (conventional) or 1.1-1.2 (modern free cooling)
• Density: 5-15 kW/rack for standard workloads, 30-50 kW/rack for AI/ML
• AI/ML cooling: liquid immersion (Submer, Asperitas), rear-door HX, direct-to-chip
• Power: A+B feeds, UPS + generator + N+1 redundancy
• Power connection: 2× power supplies per server connected to A+B PDUs

Data Center Security

• Perimeter N-S firewall (Palo Alto PA-5400/7000, FortiGate 4200F/6000)
• E-W microsegmentation (VMware NSX, Cisco ACI contracts, Illumio)
• Zero trust: EDR on every VM and identity-based policies
• DDoS protection: Radware, Arbor, and Akamai/Cloudflare cloud services
• Physical security: badge access + biometrics, CCTV, and mantraps

Orchestration

• Cisco ACI: Cisco-centric, intent-based platform
• Juniper Apstra: multi-vendor, intent-based platform
• VMware NSX-T: hypervisor overlay
• Arista CloudVision: Arista-focused platform
• Nokia SR Linux + EDA: a new challenger

Modern AI/ML Data Centers

• NVIDIA DGX H100: 8×H100 GPU, 1.5 kW, 10× 400G InfiniBand
• IB or RoCEv2 Ethernet fabric: 400G/800G spine-leaf
• High-speed storage: DDN EXAScaler, WEKA, Pure FlashBlade
• Cooling: direct liquid cooling is mandatory (>30 kW/rack)
• Orchestration: Kubernetes + GPU operator + MIG (Multi-Instance GPU)

2026 Trends

• 400G → 800G spine-leaf
• Co-packaged optics (CPO): optics integrated into the switch
• DPU (Data Processing Unit): AWS Nitro, NVIDIA BlueField, Intel IPU
• Disaggregation: white-box SONiC + NVIDIA/Broadcom ASICs
• Sustainability: PUE <1.15, renewable energy, and heat reuse

Order from OPTINOC

Turnkey data center architecture: Cisco/Juniper/Arista spine-leaf EVPN-VXLAN, certified OPTINOC 100G/400G transceivers (-50% vs. manufacturer pricing), and Apstra/ACI integration. Quotes for 20-500 racks within 48 hours.