Inference Mesh: Routing AI Workloads Across Distributed Compute

Not every AI workload needs the same compute.

Inference Mesh explores a simple systems question: if multiple devices are available, can a browser discover them, understand their capabilities, choose an appropriate peer for a workload, and execute that task remotely?

Separate discovery, routing, transport, and execution.

• Presence advertises node metadata such as CPU cores, memory, WebGPU support, compute score, and active task count.
• The router scores nodes against workload type, complexity, and hardware requirements.
• Supabase handles signaling and peer introduction.
• WebRTC DataChannels carry workload and result messages peer-to-peer.

A capable node is not automatically the best node.

The router has to balance workload requirements against device capability, latency, current load, and WebGPU availability. That turns the project from a WebRTC demo into a resource-allocation problem.

Make the transport reliable before making the router clever.

The current implementation focuses on peer discovery, WebRTC negotiation, clean separation between routing metadata and executable payloads, and remote workload/result messages.