At VMware Explore EU 2024, the session “AI Without GPUs: Using Your Existing CPU Resources to Run AI Workloads” showcased innovative approaches to AI and machine learning using CPUs. Presented by Earl Ruby from Broadcom and Keith Bradley from Nature Fresh Farms, this session emphasized the potential of leveraging Intel Xeon CPUs with Advanced Matrix Extensions (AMX) to run AI workloads efficiently without the need for GPUs.
Key Highlights:
- Introduction to AMX:
- AMX (Advanced Matrix Extensions), available in Intel’s Sapphire Rapids processors, enables high-performance matrix computations directly on CPUs, making them more capable of handling AI/ML tasks traditionally reserved for GPUs.
- Why Use CPUs for AI?:
- Cost Efficiency: Lower operating costs compared to GPUs.
- Energy Efficiency: Ideal for environments where power consumption is a concern.
- Sufficient Performance for Specific Use Cases: CPUs can efficiently handle tasks like inferencing and batch-processing ML workloads with models under 15-20 billion parameters.
- Software Stack:
- OpenVINO Toolkit: Optimizes AI/ML workloads on CPUs by compressing neural networks, improving inference performance with minimal accuracy loss.
- Intel oneAPI: Provides a unified software environment for both CPU and GPU workloads.
- Real-World Application:
- Nature Fresh Farms: Demonstrated how AI-driven automation using CPUs effectively manages complex agricultural processes, including plant lifecycle control in greenhouses.
When to Choose CPUs Over GPUs:
- Inferencing and Batch Processing: When real-time responses are not critical.
- Sustainability Goals: Lower power consumption makes CPUs a viable option.
- Cost-Conscious Environments: For scenarios where reducing operational costs is a priority.
