How Open Software and Virtualization Transform Network Operations and Business Agility
Your business units wanton-demand infrastructure for applications, and are trying public clouds. Data-intensive applications are driving demand and scale. Your company has stringent requirements for security, performance and compliance that favor your private data center for critical applications. You plan to deliver turnkey IT capabilities, already enabled for compute and underway for storage.
The network is the bottleneck. It operates at a high level of criticality with innumerable requirements for security and compliance. It’s a vast estate of heterogeneous devices managed by an array of specialized tools and administrators. Network resources take weeks to provision, and issues take days to resolve.
“The new IT infrastructure arose from the shift from expensive proprietary appliances to open operating systems on commodity hardware and the innovations of virtualization and devops”
The New Data Center Infrastructure
A transformation is underway in IT, driven by the agility, scale and cost requirements of cloud and webscale companies (Amazon, Google, Facebook) a new 'elastic' infrastructure is emerging.
The new IT infrastructure arose from the shift from expensive proprietary appliances to open operating systems on commodity hardware and the innovations of virtualization and devops.
For enterprises, it promises to transform the speed and competitiveness of business. But delivering it is the challenge. And, while compute and storage had evolved toward this vision, the network had not, until recently.
The Problem Of The Network
“The network is in my way” -James Hamilton, Architect, AWS
Networks provide the connectivity for all applications and attached devices in the enterprise, and are also the locus of the majority of the security and compliance functions of IT.
Networking is complex and highly specialized. Until recently, data center networks consisted of proprietary appliances in multi-tiered custom architectures–‘snowflakes’ – requiring specialized care and feeding.
And, due to the closed appliance nature of devices and their protocols for different operations, networks have been managed via manual commands (or brittle simulation software) and monitored via periodic polling, preventing the network from joining the new software-based operations models for agile IT.
Like compute before it, networking has also been dominated by a large incumbent vendor, Cisco, who built an industry ecosystem that separated networking from the rest of IT, suppressing innovation and maintaining market control, but is now being challenged.
The New Network
"I paint a computer with software, creating this thin soft state… Now I'm going to do that with the network," – Martin Casado, GM Networking and Security, VMWare.
With the emergence of commodity hardware, open operating systems, and virtualization in networking, the IT transforma¬tion has come to the network.
The most dramatic change is the centralization of the control plane in software that governs network traffic (network virtualization). It enables the self-service provisioning of network services by application developers, while obscuring the underlying physical network of hardware machines and cables. Its ability to virtually connect clusters also enables greater scale.
It also transforms security, decreasing the attack surface of networks and enabling highly targeted filtering and access control through the creation of 'as-many-as-you-like virtual networks'. The profound implications of this are only just emerging.
In the market, the new network platforms are solidifying. Open physical devices from Arista, Cisco, Juniper and others are proven in data center environments. Pure software operating systems and commodity hardware are growing in performance and maturity. And network virtualization platforms have become stable, scalable, and feature-rich, with strong offerings from industry leaders, increasingly viable open source alternatives and a new type of market leader in VMWare.
So, with strong platforms available, the remaining question becomes ‘Operationally, how do we do this?’
The Operations Challenge
“People forget that networking is hard,” Greg Lavender, CTO, Citi.
Early in network virtualization, the assumption was that simplified physical networks using Linux-based software would eliminate performance and scale issues and replace existing network operations with those of compute.
However, compute has an advantage in the elastic IT model: its elements are discrete. The network, as an interconnected system carrying transient payloads, bears the risk of unintended, far-reaching and unrecoverable consequences to change.
In network operations, rigid guardrails evolved to protect against potentially devastating effects. This makes real-time system-wide intelligence with deep network context – not present in compute systems - critical for network operations.
The new network model increases this criticality, changing the landscape for operations by adding de-coupled physical and virtual layers, increasing the scale, volume and temporality of elements and providing more possible paths for traffic.
With the need to gather and interpret real-time information from disparate layers of software and hardware across the network, infrastructure and applications, the information required to operate the network is no longer tractable by a human administrator, or even the periodic polling input of traditional monitoring systems.
Data-Driven Network Operations
"We want to deploy, manage, monitor and fix the network using software," Najam Ahmed, Facebook.
The real promise of the new network is operation in software.
Network operations can model other systems - Uber, hyperscale games, air traffic control - in which intelligent software dynamically abstracts masses of complex inputs into simple, intuitive user interactions supported by an ever-expanding array of intelligent underlying functions.
The foundation is real-time visibility and analytics that leverage event-driven operation, agents and micro-services to support real-time data from the infrastructure at scale. With system-wide intelligence, automation becomes possible for the network.
Achieving this requires applying the latest technologies from other industries (distributed systems, analytics, interface design) to the problems in networking. To minimize risk, automation may evolve in stages, from dynamic monitoring through to fault correction and optimization.
To get there, new operations systems will require compatibility with existing tools and processes, providing first, visibility, and then a platform for ongoing transformation- a bridge between the old and the new.
Over time, today’s network organizations will evolve into small teams that combine deep networking, software and analytics expertise to deploy expert software to meet evolving requirements. They will have far more control and visibility than today, with insight into all layers of the network, parts of the infrastructure, and all application traffic.
In that future, your network will be a resource that is intelligently managed and dynamically available to the business, with its human talent focused on driving innovation to add value.