Skip to content

support native controllers #90

@mmoskal

Description

@mmoskal

In some cases the controllers are trusted and performance is particularly important. In these cases we should be able to deploy a natively compiled controller. We would need a library that exposes AICI host interface and communicates with aicirt just like normal "wasm-runnner" does.

This is particularly important with speculative decoding

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions