Terabit Routers Prove Hocus-Pocus Solution

Posted: 06/2000

Terabit Routers Prove Hocus-Pocus Solution
By Charlotte Wolter

It will seem like magic when the quick hands of the most updated “terabit” routers work with the new generation of intelligent optical devices to create networks that can aggregate and switch huge amounts of data as waves, and provision bandwidth on demand.

The timing for introducing these new terabit routers could not be better, because the optical core is composed of a mesh of lightwaves under the mastery of hyper-intelligent optical switches and software. Formidable processing is needed at the edge of that core to tame vast flows of Internet packets into service-oriented streams that fit efficiently into lightwaves or Lambdas on dense wavelength-division multiplexed (DWDM) networks.

The new routers will not handle packet volumes anywhere near a terabit per second (tbps) in their initial deployments–200 gigabits is a likely maximum for this year. Expectations are that they will have to scale near that level within three to four years, and products are being designed with scalability in mind that ranges from 100gbps to 6.4 terabits.

Vendors predict a wave of traffic growth in the next few years that will make routing at this level necessary, based on the work of analysts in data networking.

“Some feel there will never be a need for terabit routers, which I dismiss,” says Raj
Mehta, senior analyst for RHK Inc. (www.rhk.com). “Data traffic is IP traffic, because the web and all next-generation services boil down to IP. The only way
to move IP packets is by routing. Photonic processing is at least five years out, so
the routing world really drives the optical layer, and the traffic comes from routers. As hot as optical is, the routing world is not disappearing.”

Mehta says RHK did extensive modeling of data traffic six months ago and conducted interviews with backbone service providers about traffic on their networks. The company estimates that today’s traffic is 350,000 terabytes per month and predicts that nearly will triple every year to 16 million terabytes per month in 2003.

Not everyone is convinced the core network faces an increasing wave of traffic. A critic of terabit routers concept as a product is Tom
Nolle, president of CIMI Corp., a telecommunications network consulting firm. Nolle says his firm has calculated that in the year 2004, nine terabits of “offered load traffic” will be offered to the public network. Divided among 250 standard metropolitan statistical areas
(SMSAs), this means the average traffic per SMSA is 40 gigabits.

“The point is that the devices we have today at the top end of the Cisco [Systems Inc.,
www.cisco.com] or Juniper Networks Inc. [www.juniper.net] lines are as large as we are ever going to need,” Nolle says.

While he disagrees, Mehta acknowledges, “Today there is clearly no need for a terabit router. In mid-March, the first OC-192C (concatenated) line card [for a gigabit router] was announced by Juniper, a full 10gbps on one line card. We are saying that raised the bar for the industry.”

New Core Club

Terabit routers are among the three basic components of the new optical network as it comes into focus. The first two products often are grouped with large-scale SONET multiplexes as “the optical core.”

They are: “wavelength switches” or “wave routers,” such as the Sycamore Networks Inc.
(www.sycamorenet.com) products that provide intelligent networking. These use network nodes to communicate with each other about location and bandwidth; and fault protection optical cross-connects, such as Tellium Inc.
(www.tellium.com), which switch flows of data at the wavelength level, such as OC-48s or OC-192s.

At the edge of the optical core will be the terabit routers, such as those from Avici Systems Inc.
(www.avici.com), Pluris Inc. (www.pluris.com) and Lucent Technologies Inc.
(www.lucent.com). Although they are technically electrical products, their main task is to groom IP traffic from many sources into flows destined for lambdas and to route those flows to the correct optical port.

The difference between routers and switches is that routers examine every packet, looking for its identity, destination, contents and, soon, its priority. Switches route whole flows of traffic based on information about the entire flow, not its contents.

Today, none of these devices does what its name implies. Waves cannot be switched directly. They must be converted to electricity for switching. The
"photonic" cross-connects Lucent and Agilent Technologies Inc. (www.agilent.com) introduced earlier this year do not have to do an optical-electronic-optical
(OEO) conversion. But they also are not complete switches.

“Optical” cross-connects, such as Tellium, convert traffic to electricity for switching purposes. But no router deployed today is at terabit levels, and a 200gbps or 300gbps device may be the largest deployment this year.

However, these products are the first implementations of a new architecture for optical core networks. Whether it is switched optically or electrically, the basic transport unit is now the wavelength or “lambda,” not the SONET mux.

Networks are constructed as meshes rather than rings, and each “node” is an intersection of multiple paths. This means that having alternate routes for traffic maintains protection.

And, as Sycamore pioneered, the nodes on the core network are intelligent and can communicate with each other. This enables operators, to find alternate routes for protection, and to provision bandwidth automatically wherever capacity exists, reducing provisioning times from months to days or even hours. In March, Ciena Corp.
(www.ciena.com) demonstrated a feature called Optical Priority Provisioning that enables any-priority, end-to-end, real-time service provisioning implementation within minutes.

The role of large routers that sit at the edge of these networks also changes. Routers will aggregate traffic for lambdas and will communicate directly with core optical networks to provision bandwidth for the flows they create.

MPLS Takes Charge

An important enabling mechanism for this will be an extension of multiprotocol label switching (MPLS), an IP mechanism for creating different levels of quality of service (QoS). MPLS will be employed as multiprotocol lambda switching, with MPLS signaling used to label whole lambdas rather than individual packets.

In March, a new industry group, the MPLS Forum
(www.mplsforum.com), was formed to promote MPLS in this new role.

Lambda MPLS can maximize the use of waves in a lambda-routed network. The network has lambdas running among nodes, but, “you can’t really put enough waves on a fiber to create a complete mesh of service points of presence through the optical core,” Nolle says. “So you use MPLS label-switched paths as virtual wavelengths.”

Vivace Networks Inc.
(www.vivacenetworks.com) will use lambda MPLS capability in its large-scale router device late this year.

“Our view is that you have a service-initiation point, which is where you take slower traffic and aggregate it into lambdas,” says CEO Ken Koenig. “Even better is to aggregate it into MPLS flows, which look like virtual paths, and aggregate those onto a lambda. You get better utilization of that lambda’s bandwidth.”

Using lambda MPLS makes it possible to give each virtual path its own character based on size, and whether the bandwidth
is expandable.

“If it is a voice connection in an MPLS path, it looks like a concrete pipe going through and is always the same size,” says Koenig. “But another pipe [MPLS path] may be like rubber because, if you get data in bursts, it can shrink or grow.”

It is possible to have both kinds of traffic sharing the same lambda and providing for many types of service initiation across the backbone.

The point of using MPLS in this manner is for the services, “taking large hunks of bandwidth for a carrier to convert to hunks of revenue in forms that they understand, such as TDM [time-division multiplexing] voice and ATM and leased line,” explains Mike Kazban, vice president, marketing and business development for Vivace.

“It also gives the infrastructure to do advanced IP services that haven’t been thought of yet,” he adds.

The “routers” that will do this still may be called routers, “but MPLS is creating new devices that have switching and routing capabilities and do service provision,” says Kazban.

“It is taking the best of the circuit-switched world and the best of the packet world, doing service provisioning product that has Layer 2 and Layer 3 at the same time,” Kazban says.

Unlike today’s routers, these devices will know MPLS paths and will police different service levels. They will allow a service provider to set up “virtual service networks over virtual fiber infrastructure and each has virtual services,” says Kazban.

A customer could get a service mix that includes 1mbps of TDM voice, 1mbps of variable-bit-rate service for video and 2mbps of best-effort bandwidth. And, service providers could move away from transporting bits to providing service mixes.

Vendors say that MPLS will evolve IP networks from their current best-effort capabilities.

“They say it will make IP more ATM-like by attaching connection information to the packet,” Mehta says. “It sets up virtual trunks. … IP is going to bear many of the quality of service mechanisms that ATM does now.”

Talking Optical

The MPLS developments make it more important that routers communicate with optical networks beyond presenting an aggregated traffic flow. One effort is the Optical Domain Service Interconnect (ODSI) initiative, which Sycamore launched earlier this year. The effort would develop open interfaces and signaling protocols among service devices, such as IP routers, SONET add/drop muxes, ATM switches, and core optical equipment.

The electrical devices could request bandwidth dynamically from the optical network core.

“The issue is for a router to be able to communicate dynamically with the network and command bandwidth, to be able to provision bandwidth automatically,” Mehta says. “There is definite value there” for automating provisioning.

That the packet-switching and optical core networks worlds even talk to each other is a first for the industry.

“Where the real value is going to come out is in companies with data networking expertise that also understand optical markets,” says Rick Thompson, senior manager, product marketing for Sycamore Networks.

The Routers

The number of announced router vendors aiming for terabits speeds is small, but is expected to grow significantly this year.

Among the leading vendors is Avici Systems, whose TSR router underwent a series of performance tests with an independent laboratory in April.

The router demonstrated it could maintain a route table of more than 300,000 different routes. It also displayed a forwarding rate of 160,000 line routes with 22-byte headers at line rates–about twice the speed demonstrated by any vendor so far.

Today’s Internet has about 70,000 entries in a route table.

“The testing is significant because it is a validation of the performance and architecture they have,” said Mehta.

IronBridge Networks Inc.
(www.ironbridgenetworks.com) is a relative newcomer to the terabit router space. The company has not yet announced its product, but one feature is expected to be the ability to differentiate between priority and best-effort traffic, “and to use links that are normally held in reserve for protection to transmit best-effort traffic,” says Doug
Antaya, vice president of marketing. If the link is needed for protection, best-effort traffic can be delayed.

Lucent, which acquired startup Nexabit as its terabit router platform, is moving its product toward increasing numbers of OC-192 ports to meet the growing capabilities of DWDM networks, which now have 40 to 80 channels of OC-192 traffic, says Ram Krishna, director of internetworking, Lucent Core IP Routing.

The company, which calls its device a router/switcher, considers it “a multiservice box,” operating at Layer 2 and Layer 3 of a network, says Krishna.

It can do frame-relay switching and MPLS switching on the same line card. A key feature is its ability to swap out operating software, which is in a separate module, without having to bring the router down.

Pluris announced last year it would bring its product to market during the fourth quarter. This will be two chassis, with one entry-level size and one large, highly scalable product. The chassis are populated with three kinds of cards: line cards that provide interfaces; shelf cards that do system timing and management and run routing protocols; and switching-fabric cards.

Each line card has 10gbps of port capacity that can be allocated as one OC-192, four OC-48s or 16 OC-12s or several gigabit Ethernets. Each fabric card has 90gbps of switching capacity, so 16 fabric cards in a single chassis will deliver 1.5 terabits of capacity.

Vivace Networks has not announced officially, but the company plans to introduce a product that uses MPLS to provide service-router-type functions.

Casting Doubts

Besides pooh-poohing core traffic estimates of multiple terabits, Nolle maintains that the idea of a terabit router, particularly one that works as today’s routers work,
is flawed.

First, the amount of traffic represented by one metro access group–roughly the size of a local calling area, which would feed such a router–isn’t anything like a terabit, says Nolle.

“The largest amount that we can see is something on the order of a quarter of a terabit,” he says. “Before you collect more traffic than that, it would be sensible to use multiple access connections and efficiently divide up customers.”

Nolle also points out, the advantage that a lambda-routed core network brings–the intelligence for every node to communicate to every other node–means every destination is a hop away from the originating node, not multiple hops as in a typical Internet backbone.

Edge to edge is always one hop. Thus, a key router function of finding the best route across a network is not needed.

“The forwarding device of the future will not resemble the forwarding behavior of routers today,” Nolle says.

For the same reason, there will be no need for router devices to “discover” the network’s topology, as they do today.

“Most of the processing and logic and feature differentiation in the router space arises through the process of topology discovery. There isn’t anything to discover in an optical network. It is opaque and every router appears to be directly connected to any other node,” he says.

Instead, Nolle foresees a product he calls SPoPs (service points of presence) that take services from the access network and forwards them through the core, in the mold of Tenor networks.

He estimates that as many as 20 vendors may announce products during the year that are in this product category. Important to these is MPLS for lambdas, which creates paths through the network that are virtual wavelengths, which increases the number of discrete “waves” available for traffic.

Future devices will differ in the way they adjust features to accommodate the concept of ATM access concentration and ATM core networks, Nolle says.

Charlotte Wolter is infrastructure editor for PHONE+ magazine.