Energy Concerns with HPC Systems and Applications

Technical Deep Dive: Energy Concerns with HPC Systems and Applications

Overview

This paper surveys the landscape of energy concerns in computer systems, including both hardware and software aspects. The focus is on energy efficiency in high-performance computing (HPC) systems, embedded systems, accelerators, and modern supercomputers.

Problem & Context

• Energy consumption has become a critical concern in computer systems, driven by factors like climate change and the growing demand for intelligent devices and HPC applications.
• For HPC systems, energy efficiency is a major challenge, as the electricity bill increasingly dominates the overall operating costs.
• Embedded systems and mobile platforms also face energy constraints due to their limited battery power.
• The carbon footprint of computing, especially for AI applications, is a growing environmental concern.

Methodology

• The paper reviews existing surveys on energy management in HPC and embedded systems.
• It provides a quantitative overview of key energy metrics in the supercomputer ecosystem, from power usage effectiveness (PUE) to carbon footprint.
• It surveys the energy characteristics and optimization techniques for various hardware components, including CPUs, GPUs, TPUs, FPGAs, and microcontrollers.
• The paper also catalogues a wide range of energy management tools targeting different hardware platforms.
• Finally, it examines energy optimization techniques, both static (e.g., hardware design, programming practices) and dynamic (e.g., DVFS, power capping, workload consolidation).

Energy Metrics

Supercomputer Ecosystem

• The top supercomputers now consume 1-10 MW of power on average, equivalent to the electricity needs of 10,000 homes.
• The electricity bill for the Fugaku supercomputer in 2020 was nearly $60 million.
• Metrics like thermal design power (TDP), average CPU power (ACP), and space-wattage-performance (SWaP) are used to characterize energy efficiency.
• Power usage effectiveness (PUE) and data center infrastructure efficiency (DCiE) are common macroscopic metrics for data center energy efficiency.
• Carbon footprint can be estimated from power consumption using the carbon intensity of the energy supply.

Accelerators

• GPUs are highly efficient for parallel workloads, with performance-per-watt ratios superior to general-purpose CPUs.
• Modern GPUs like AMD Instinct MI250X and NVIDIA H100 have TDPs of 500-700W.
• TPUs can perform 4 TFLOPS at only 2W, demonstrating extreme energy efficiency for neural network workloads.
• FPGAs are becoming more prominent in HPC, but their energy efficiency is still an important concern.

Embedded Systems

• Microcontrollers like Arduino, Raspberry Pi, and Coral Dev Board are highly energy-efficient, with power consumption in the range of 0.5-10W.
• These low-power systems are considered good candidates for energy-constrained applications.

General-Purpose Processors

• x86-based CPUs from Intel and AMD still dominate the HPC landscape, with a focus on energy-efficient designs.
• ARM-based processors are also emerging as energy-efficient alternatives for HPC and embedded applications.

Energy Management Tools

The paper surveys a wide range of energy management tools targeting different hardware platforms:

GPUs

• NVIDIA-SMI and ROCM-SMI provide power management, monitoring, and optimization capabilities for NVIDIA and AMD GPUs, respectively.

CPUs

• Intel RAPL, AMD RAPL, and hardware-specific tools like tx2mon provide access to energy measurement and control interfaces.
• Software tools like PAPI, Perf, PowerTOP, PyJoules, and Intel Power Gadget enable energy profiling and optimization.

Embedded Systems

• Benchmarks like EEMBC CoreMark-Pro and ULPMark, as well as tools like Dr. Wattson and PSoC 5LP, enable energy measurement and optimization for microcontrollers.

Modern HPC Systems

• Hardware-based tools like HDEEM, WattProf, and DiG provide high-frequency power monitoring for individual compute nodes.
• Software frameworks like GEOPM, EAR, READEX, and MERIC enable dynamic energy management and optimization for HPC applications.

Energy Optimization Techniques

The paper categorizes energy optimization techniques into static and dynamic approaches:

Static Approaches

• Hybrid CPU designs (e.g., big.LITTLE, Alder Lake)
• Optimization of programming languages and best practices
• Machine learning-based prediction models for energy and PUE
• Energy-aware hardware design and architectures
• Analytical models for energy-aware scheduling and memory management

Dynamic Approaches

• Dynamic power management (DPM) techniques, including DVFS and power capping
• Workload consolidation and on/off policies for power management
• Hybrid approaches combining static and dynamic techniques

Energy Concerns for AI Applications

• Training and running large AI models, such as GPT-3, can have significant energy and carbon footprints.
• Studies have estimated the carbon footprint of AI training and deployment, highlighting the need for energy-efficient algorithms, hardware, and infrastructure.
• Tools like ML CO2 Impact, Green Algorithms, CodeCarbon, Tracarbon, and Eco2AI enable estimating the energy and carbon footprint of AI workloads.
• Optimization techniques for AI applications include quantization, pruning, filter compression, neural architecture search, and knowledge distillation.

Conclusion

• Energy concerns are crucial for computer systems due to the growing costs and environmental impact of computing.
• Addressing energy efficiency requires holistic approaches across hardware, software, and system levels.
• Continued research and development in energy-aware architectures, optimization techniques, and management tools are essential to meet the energy and carbon challenges faced by modern computing systems.