GRAFT: gRPC-Routed Agent Framework for Tasking in Edge and Personal Devices
Chinmay Shringi (New York University), Alon Hillel-Tuch (New York University), Sariya Rizwan (Pace University)
Architectural Patterns & Composition System Optimization & Efficiency
Summary
A distributed edge orchestration system that routes structured tasks across heterogeneous personal devices using gRPC, enabling multi-device SLM workloads entirely off-grid.
Description
GRAFT is a distributed edge orchestration system that routes structured tasks across heterogeneous personal devices, enabling workloads that exceed the capability of any single small language model (SLM) to be completed entirely off-grid without cloud services. The system provides a sessioned gRPC control plane with device registration, capability-aware policy routing, compute-target inference dispatch (CPU, GPU, NPU), safety-bounded remote execution, and per-request telemetry. We demonstrate GRAFT through a multi-device PDF summarization workflow in which a coordinator partitions a document across four devices, each running a local SLM, and merges partial results through a staged execution plan with explicit synchronization barriers. Our evaluation on a four-device mesh covering silicon from Apple (laptop), Snapdragon X Elite (laptop), Samsung (mobile), and Arduino Q (embedded) hardware shows that all planned tasks execute on their intended devices, stage ordering is preserved, and end-to-end completion time is governed by the slowest parallel worker rather than total document size. The live demonstration exposes routing decisions, per-task progress, device-level timing, and failure semantics in real time, contributing both a novel systems architecture for multi-device agent orchestration and a compelling interactive experience for conference attendees.
