Poby: SmartNIC-accelerated Image Provisioning for Coldstart in Clouds

The Coldstart Challenge in Cloud Computing

• 💡 The coldstart problem in cloud platforms leads to significant latency, particularly for serverless computing, degrading user experience.
• 📊 Image provisioning, encompassing download and extraction, is identified as the most time-consuming step, often exceeding 72% of total coldstart time.
• ⚠️ Existing strategies like warm start or snapshot recovery are often ineffective or introduce prohibitive overheads, failing to fully address the core issue.

Limitations of Current Solutions

• 🔍 Many efforts to accelerate coldstart focus on image download, overlooking the critical and resource-intensive image extraction process.
• 📈 Experiments show extraction accounts for over 68% of image provisioning time, making it a primary performance bottleneck and causing high tail latency when co-located with other containers.
• ❌ Previous avoidance-based strategies impose unrealistic demands like accurate request prediction or proactive caching of all container snapshots, which are difficult to fulfill in real-world systems.

Introducing Poby: SmartNIC-Accelerated Provisioning

• 🚀 Poby is a novel software-hardware collaborative system designed to accelerate image provisioning by leveraging SmartNICs.
• 💡 SmartNICs, with their general-purpose SOCs and domain-specific hardware accelerators, provide a promising approach to offload and speed up computing tasks.
• ⚡ Poby addresses key challenges including the offloading dilemma, serial execution of operations, and performance bottlenecks of centralized image registries.

Poby's Innovative Design Principles

• 🧩 Poby utilizes a disaggregated architecture that offloads decompression to SmartNIC hardware accelerators while deferring unpacking to the host to optimize data transfer.
• ⚙️ It implements a pipeline-based, data-driven workflow with block-based redundant pipelines to eliminate delays from serial execution and maximize hardware parallelism.
• 🌐 A best-effort distributed image download scheme is integrated to minimize overheads and alleviate the performance bottlenecks associated with conventional centralized registries.

Performance and Resource Efficiency

• ✅ Poby demonstrates significant performance improvements, achieving an average speedup of 11.5 times over containerd and 7.1 times over iSulad across various workloads.
• 📊 The system effectively reduces host CPU usage by 87.5%, by offloading 55.3% of user-mode CPU usage to the SmartNIC.
• 🚀 Poby also shows superior concurrency and comparable scalability to state-of-the-art systems with nearly zero overhead.