Power Density Challenges Of Next Generation Telecommunication Networks

Despite the current downturn in the telecommunications industry, data transfer (i.e. network traffic) is continuing to grow at a 100% rate [1]. It is already projected that network traffic demand will reach tens of Tb/s in a couple of years from now (see predicted traffic demands from telecommunications analysts Ryan, Hankin & Kent, Figure 1). At the same time, network power requirements will scale with Internet growth. However, power demand may slow network growth.

Figure 1. Prediction of traffic growth till 2005 (PoP = point of presence, place where equipment is located; some data extracted from [1]).In response to the projected network growth, the industry has to create a new design and architecture space to address the power-density challenges of next generation telecommunication networks. There are two aspects of this issue: network architecture and telecommunications equipment design. Each requires some elaboration.

Most network power is consumed by providing the functionality for transport, switching, routing, processing and storage. Minimizing the total power per network throughput is the main architectural goal. The achievement of this goal depends heavily on the ability to keep transferring the bits in the optical domain from the point that they enter a network to the point that they leave, bringing a clear benefit from the power density standpoint. The so-called all-optical network (AON) promises to be an enabler in developing a “less power hungry” network, with the added benefit of being able to cope successfully with the expected network growth and stringent market-driven cost reduction requirements.

From the equipment design standpoint, the first step should be to explore techniques and technologies for power per bit reductions for multi-terabit transport, processing, routing, storage and switching by integrating state-of-the-art industry advances in high-density packaging and high-speed interconnects. At the same time, a key challenge of high-density, high-speed equipment is thermal management.

The focus of this article is to define a new network power efficiency paradigm that can feasibly be used to re-architect today’s opaque communication network (consisting of multi-wavelength links with electrical processing at their ends) to a transparent AON where data is transmitted, managed and monitored in the optical domain. In addition, the article emphasizes two critical elements of power efficient and feasible telecommunications equipment design: power reduction strategies and high-density thermal management.

Network Architecture – Role of All-Optical Network

The network architecture will accommodate a variety of applications using voice, data, “on-demand” news, movies, music, research, DVD-quality video and other emerging multimedia services. The network architects currently cope with increasing capacity requirements to keep up with network traffic growth. This session points out power density benefits of an AON vs. today’s network as well as some directions leading to “less power hungry” network.

To address the capacity demands, the world of telecommunications is rapidly moving from copper wire to fiber optics based communication networks, which are presently dominating in data transmission. One of the most disruptive optical technologies of the 1990’s is surely WDM (wavelength division multiplexing), and in particular, DWDM (dense-WDM). In networks based on WDM, wavelengths are treated as pipes that can each be filled with an arbitrary traffic type, whether voice, data, transactions, network protocol information, video, or something else.

Today, the deployment of DWDM has been primary restricted to point-to-point WDM systems, providing increased network capacity. Within a DWDM system, the optical signal is carried mainly in the optical domain. However, due to the degradation of the signal along the path of systems deployed today, the signal needs to be regenerated (every 400 – 600 km). Signal regeneration requires conversion of the optical signal to the electrical domain for processing and a transfer back to the optical domain (Optical to Electronic to Optical i.e. O/E/O conversion). According to some studies, almost 50% of the power consumed in transport networks is consumed in nodes where regeneration is required. Therefore, an architectural goal is to reduce or eliminate the need for signal regeneration. There are also the other drivers, such as cost reduction, that support the same goal: elimination or reduction of signal regenerators in a network.

Additionally, wavelengths are switched at cross-connects, almost all having an electrical core today. Cross-connects with a transparent photonic (optical) core is still a hot topic of research and development. The aim of a transparent photonic cross-connect is to pass transport photons right through without converting them to electrical signals. Further into the future, individual wavelengths will be converted to other wavelengths using tunable wavelength-converting optoelectronics, perhaps based on semiconductor optical amplifiers; again, no electronic signals at the high data rates will be involved. Coupled with fast-acting protocols that respond dynamically to network traffic demands, these innovations will deliver the agile AON with significantly less power per bit transferred [2]. It is obviously an advantage that makes the AON attractive from a power density perspective. However, it is just a part of the equation. The communication equipment design, including thermal management, is the other one.

Communication Equipment Design

At the equipment level, architecture and design objectives are compelled increasingly to focus on power reduction strategies, which exceed the conventional electrical capacity that follows Moore’s Law (Figure 2). From a power and cost perspective, the packaging and interconnect challenges are to minimize the total number of interconnects and maximize shelf, board and chip I/O and packaging density. The focus of this section is on design issues associated with high speed, high density packaging and interconnects. A proposed solution space for capacity growth appears in Figure 2.

Figure 2. Proposed solution space for capacity growth. (PoP = point of presence)In switching/routing nodes where many shelves must be interconnected to achieve multi-terabit switch fabrics, considerable power is expended in interconnecting the shelves. In tens of Tb/s switch fabrics within a node, shelf-to-shelf fiber interconnects account for up to 40% of the fabric power consumed. Reducing the number of shelves by increasing shelf packaging density and interconnect speed can dramatically reduce fabric power.

High speed, short haul parallel optical interconnects based on Vertical Cavity Surface Emitting Laser (VCSEL) technology offer significant power and density advantages within a multi-terabit node. VCSELs typically dissipate only 20 mW per channel when operated at eye-safe optical levels. VCSEL transceivers offer very dense interconnects. There are also wide-WDM solutions that deploy more VCSELs with wavelengths that are on a 5 to 10nm grid, all launched into one fiber. Unlike dense-WDM, wide-WDM does not need thermoelectric coolers and complex stabilization circuits. Nonetheless, presently available VCSELs only tolerate approximately 0 to 80^oC junction temperatures with technology efforts to increase the range to 110^oC well underway.

Following the recent wave of System-on-Chip functional integration, analogous power reductions can be achieved by minimizing board-to-board interconnects. Extending the line rates of backplane copper interconnects without expensive connectors will help power efficiency. This points towards high-density packaging and high speed PCB level copper interconnects. High throughput shelves have two challenges once the silicon and optics are developed. They need a range of new high density PCB, passive, device package, thermal, power conversion and EM technologies, and these technologies must be feasibly integrated into circuit packs and serviceable modules.

PCB level packaging [3,4] is continuing towards using fewer components per functional block, interconnecting finer pitch devices and using boards with dual-side active components along with more direct chip attachment. To reduce board real estate, fine-pitch standards will decrease, pending manufacturability improvements. Leaders in the communication industry use embedded passives that are revolutionizing I/O density by routing and terminating 400+ pin devices within their footprint.

Deep sub-micron technologies, copper interconnects and low voltage operation are continuing to evolve silicon at Moore’s Law rates. SiGe offers some relief with regards to power per function at high clock rates. To keep pace with device technology, IC packaging technologies are evolving from plastic quad flat packs (PQFP) and small outline ICs (SOIC) to chip scale packages (CSP), multichip modules (MCM), ball grid arrays (BGA), micro ball grid arrays (�BGA) and direct chip attach (DCA) technologies [4,5].

Serialized GHz interconnect line rates are addressing part of the I/O density challenge while offering the chance to reduce overall power consumed by chip-to-chip communication. As resistance accounts for less and less of the interconnect impedance as the frequency increases in the Gb/s range, the line rate versus the number of lines should be optimized for total power [3]. Technologies such as blind and buried vias, and embedded passives have been successfully applied to improve power efficiency. Advanced PCB materials, such as new laminates, silicon, glass on ceramics, etc., can also reduce the line drive power.

With these power reductions and I/O density improvements, the net outcome is still increased thermal and EM field density. The challenge then becomes how to deal reliably with the significant power consumed by the ICs and opto-electronic devices and to contain the EM fields. As GHz circuits are likely to need fully enclosed PCBs, the challenge is shifting to thermal management.

Role of Thermal Management

Thermal management technologies continue to evolve rapidly due to the market requirements for high chip heat fluxes, densely populated boards, multi-kW shelves, high-density frames, and harsh outdoor conditions (for wireless equipment deployment). More stringent cost, performance and reliability constraints make thermal management one of the key enabling technologies in the development of equipment for next generation telecommunication networks.

Conventional thermal techniques, used for decades, are very close to their performance limits. New techniques and materials, including such cooling technologies as micro/macro heat pipes, liquid cooling (subject to numerous discussions), thermoelectric coolers, thermal spreaders and interfaces will have to be further developed and implemented in real applications. Thermal management challenges increasingly must be addressed:

• Heat removal from high flux chips/modules within a reducing volume and with low pressure drop.
• Heat spreading from relatively small areas to large heat sinks or conduction plates.
• Thermal integration with EMI for high speed, low voltage circuits.
• Noise management with increasing air velocities.
• Sophisticated tools for modeling and testing.
• Photonic packaging for sensitive laser sources including precise temperature control techniques.
• New interface materials for conduction optimization of increasingly small chips.

 

By successfully overcoming the challenges, thermal management will contribute to the increased competitiveness of packaged communication products. Further advances and developments are projected in the following cooling technologies: thermal spreaders, heat pipes, water cooling, thermal interfaces, thermoelectric cooling and direct immersion cooling [6,7].

Consequently, future thermal management, when addressed within the above constraints, must combine the best features of conventional, low cost methods with disruptive thermal solutions to satisfy the requirements of the next decade’s products.

Conclusion

With projected node sizes from hundreds of Gb/s to tens of Tb/s, it can be extrapolated from current architectures that hundreds of kilowatts of power per node will be consumed. Electrical functionality is the power density bottleneck of next generation optical networks. Power demands may limit network growth.

Maximizing the use of optical technologies (WDM) in network architectures, improvements in optical amplifiers and long-haul line transport systems, deployment of photonic cross-connects (i.e. focusing on agile optical networks), will ultimately deliver the lowest power per function for transport and wavelength switching in tomorrow’s networks.

However, electrical packet-level switching, storage and processing will continue to increase power demands for the foreseeable future, particularly on IP networks. To accelerate the reduction in power-per-function beyond Moore’s Law, industry is investing in key technologies that complement the industry base (dense PCB technologies, low power devices, high speed copper and optical interconnects and advanced thermal concepts) and that show both reduced power and cost.

Therefore, power-density challenges of next generation optical networks are of two natures that will require development in parallel: network and system architecture, and power per function reduction combined with enhanced thermal management.

References 

1. Ryan, J., “The Business of Telecomm”, invited presentation at 18th World Telecomm Congress, WTC/ISS 2002, Paris, FR, September 2002.
2. Vukovic, A., “All-Optical Networking – Today’s Challenges”, Ottawa Photonic Research Alliance Workshop, May 2001.
3. Suhir, E., “The Future of Microelectronics and Photonics and the Role of Mechanics and Materials”, IEEE/CPMT Electronics Packaging Technology Conference, 1997.
4. Tummala, R., “3rd Generation of Packaging Beyond MCM, CSP, Flip-chip and Micro-via Board Technologies”, IEEE/CPMT Electronics Packaging Technology Conference, 1998.
5. Bar-Cohen, A., “Thermal Management of Microelectronics in 21st Century”, IEEE/CPMT Electronics Packaging Technology Conference, 1997.
6. Vukovic, A., Watkins, J., Vukovic, M., “Power Density Challenges of Next Generation IP Networks”, 17th World Telecomm Congress WTC/ISS 2000, Birmingham, UK, May 2000.
7. 2000 NEMI Roadmap, National Electronic Manufacturing Initiative, Inc., Herndon, VA, 2000.”