Recap: InfiniEdge AI Release 3.0 Code Lab – building the future of edge AI with SPEAR

Introduction

On January 17 the InfiniEdge AI community gathered at the Stevenson Event Center on the University of California Santa Cruz campus for a hands‑on code lab celebrating the Release 3.0 of InfiniEdge AI. The event was organized by the LF Edge AI & Data (LF Edge AI) project, with Tina Tsou giving a preview of the release and Wilson Wang (ByteDance Global Edge Team) leading a deep dive into SPEAR, the new serverless AI‑agent framework. In this blog we recap the key moments, lessons and demos from the session.

Release preview & roadmap

Tina Tsou opened the event with an overview of the new features and roadmap for InfiniEdge AI 3.0. Her talk framed AI agents as a “burger” stack that orchestrates between models, hardware and users — the top bun is the user‑facing interface, the bottom bun is the edge hardware and runtime, and the meat patties are the generative models and tools. She emphasised that many existing AI‑agent platforms are proprietary and run exclusively in cloud datacentres, leading to vendor lock‑in and heavy, monolithic frameworks. InfiniEdge AI 3.0 aims to fix this by prioritising open source, flexibility and on‑premises support. The roadmap includes tighter integration with edge devices, improved hardware abstraction layers, and more developer‑friendly tooling.

Why a new AI‑agent framework?

Wilson Wang set the stage by summarising the pain points in current AI‑agent development. Proprietary platforms force developers into closed ecosystems with limited extensibility; many solutions are heavyweight, rely on high‑code frameworks and run only in the public cloud. There is a growing demand for low‑code agent frameworks that can run on‑premises or at the edge, support multiple runtimes and avoid vendor lock‑in. This context motivated SPEAR (Serverless Platform for Edge AI Runtime), part of InfiniEdge Workstream 3.

Introducing SPEAR

SPEAR is a distributed, open‑source AI‑agent serverless framework that enables developers to build and run agents on edge devices or private clouds. It supports multiple runtimes, including WebAssembly (WASM), containers and native processes, so workloads can be executed in the most appropriate environment. The framework enables distributed execution, automatically scheduling tasks across a cluster of nodes to improve performance and fault tolerance. It is also lightweight and secure, using WASM and a HostCall API to allow custom functions to call host resources safely. The project is open source to avoid vendor lock‑in and invites contributions from the community.

Architecture at a glance

The architecture comprises three key components:

1. Metadata server: stores agent definitions, tasks and versions, and exposes APIs for management.

2. SPEARlet: a small runtime that loads agent metadata, manages scheduling and state, and communicates with the metadata server. It shares the same memory space as the user functions for high performance and uses a HostCall controller to safely call host functions.

3. Worker pods: execute individual tasks in WASM modules, containers or processes. They use an object store to persist intermediate results and controllers (Task and Node controllers) to manage scheduling and resource allocation.

This design allows agents to be defined in a declarative YAML file and executed across nodes. Developers can register new tasks and plug in custom functions without rewriting core services.

Live demo & hands‑on practice

After introducing SPEAR, Wilson led a live demo showing how to build a simple AI agent to summarise text. The entire agent, from YAML definition to Python tasks, spanned around 100 lines of code. He demonstrated how SPEAR’s scheduler automatically distributes work and how the HostCall controller safely invokes Python functions. The audience then participated in a hands‑on lab where they adapted the demo to build their own agents for language translation, image classification and sensor data processing. Developers appreciated the clear separation between the declarative specification and the runtime code, and they were able to run agents on both their laptops and edge servers without changes.

Hardware platform: SuperMicro SYS‑212B‑FN4TP

To power the hands‑on exercises, the code lab used a SuperMicro SYS‑212B‑FN4TP edge server. This 2U short‑depth system features front I/O with four 10 GbE ports and optional hot‑swap NVMe bays, making it easy to deploy in space‑constrained environments. Inside, it supports single‑socket Intel Xeon 6700/6500‑series processors and up to three single‑width GPUs, providing the compute horsepower needed for AI inference and machine‑learning workloads at the edge. With up to 2 TB of DDR5 memory, multiple PCIe 5.0 expansion slots and onboard 10 GbE networking, the platform delivered smooth, real‑time performance for running SPEAR agents during the lab. Supermicro positions this system for AI inference, machine learning, cloud computing and 5G networking applications, so it was a natural choice for the InfiniEdge AI workshop.

Key takeaways

• Open source & vendor‑neutral: SPEAR avoids vendor lock‑in by being fully open source and encouraging community contributions.

• Flexible runtimes: Support for WebAssembly, containers and native processes lets developers choose the right execution environment for each task.

• Distributed & lightweight: The framework automatically distributes tasks across nodes while remaining lightweight, enabling near real‑time responses on resource‑constrained devices.

• Declarative & low‑code: Agents are defined declaratively in YAML and tasks can be written in just a few lines of Python, reducing complexity and accelerating development.

Conclusion

The InfiniEdge AI Release 3.0 Code Lab showcased how open‑source tooling can bring AI agents closer to edge devices. By combining a clear roadmap, a strong community and the novel SPEAR framework, the project aims to make edge AI development more accessible and vendor‑neutral. If you couldn’t attend the code lab, the SPEAR project is available on GitHub, and the organisers encourage everyone to experiment, contribute and help shape the future of edge AI. We look forward to seeing what the community builds next!