[stack] omnium-gatherum

.claude/plans/e-on-vat-objects.md

@@ -0,0 +1,159 @@
+# Plan: Enable E() Usage on Vat Objects from Background
+
+## Overview
+
+Bridge CapTP slots to kernel krefs, enabling `E()` usage on any kernel object reference from the extension background. This uses CapTP's documented extension point `makeCapTPImportExportTables` to intercept slot resolution and create presences backed by krefs that route through `kernel.queueMessage()`.
+
+## Key Insight
+
+The kernel already has `kernel-marshal.ts` that demonstrates the kref↔marshal bridging pattern with `kslot()` and `krefOf()`. We apply the same pattern to CapTP's slot system.
+
+## Architecture
+
+```
+Background                    Kernel Worker
+   │                              │
+   │  E(presence).method(args)    │
+   │  ────────────────────────►   │
+   │  (kref in slot, method call) │
+   │                              │
+   │                              │  queueMessage(kref, method, args)
+   │                              │  ────────────────────────────►
+   │                              │                               Vat
+   │  result with krefs           │
+   │  ◄────────────────────────   │
+   │  (auto-wrapped as presences) │
+```
+
+## Implementation Phases
+
+### Phase 1: Kref-Aware Background CapTP
+
+**Files:** `packages/kernel-browser-runtime/src/background-captp.ts`
+
+1. Create `makeKrefImportExportTables()` function:
+
+   - `exportSlot(obj)`: If obj is a kref presence, return the kref string
+   - `importSlot(slot)`: If slot is a kref string, create/return a presence
+
+2. Create `makeKrefPresence(kref, sendToKernel)` factory:
+
+   - Uses `resolveWithPresence(handler)` from `@endo/promise-kit`
+   - Handler routes `GET`, `CALL`, `SEND` through kernel
+   - Caches presences by kref to ensure identity stability
+
+3. Modify `makeBackgroundCapTP()`:
+   - Accept `makeCapTPImportExportTables` option
+   - Wire up kref tables to CapTP instance
+
+**Key Code Pattern:**
+
+```typescript
+function makeKrefPresence(kref: string, sendToKernel: SendFn): object {
+  const { resolve, promise } = makePromiseKit();
+  resolve(
+    resolveWithPresence({
+      applyMethod(_target, method, args) {
+        return sendToKernel('queueMessage', { target: kref, method, args });
+      },
+    }),
+  );
+  return promise;
+}
+```
+
+### Phase 2: Kernel-Side Kref Serialization
+
+**Files:** `packages/kernel-browser-runtime/src/kernel-worker/captp/kernel-captp.ts`
+
+1. Modify kernel CapTP to use kref-aware slot tables
+2. When serializing results, convert kernel objects to kref strings
+3. When deserializing arguments, convert kref strings to kernel dispatch targets
+
+### Phase 3: Public API
+
+**Files:** `packages/kernel-browser-runtime/src/background-captp.ts`
+
+Export utilities:
+
+- `resolveKref(kref: string): Promise<object>` - Get E()-usable presence for a kref
+- `isKrefPresence(obj: unknown): boolean` - Type guard
+- `krefOf(presence: object): string | undefined` - Extract kref from presence
+
+### Phase 4: Promise Kref Handling
+
+**Files:** Background and kernel CapTP files
+
+1. Handle `kp*` (kernel promise) krefs specially
+2. Subscribe to promise resolution via kernel
+3. Forward resolution/rejection to background promise
+4. Add `subscribePromise(kpref)` to KernelFacade
+
+### Phase 5: Argument Serialization
+
+**Files:** Background CapTP
+
+1. When calling `E(presence).method(arg1, arg2)`, serialize args through kref tables
+2. Local objects passed as args need special handling (potential future export)
+3. For Phase 1, only support passing kref presences and primitives as arguments
+
+### Phase 6: Garbage Collection
+
+**Files:** Background CapTP, KernelFacade
+
+1. Use `FinalizationRegistry` to detect when presences are GC'd
+2. Batch and send `dropKref(kref)` to kernel
+3. Add `dropKref(kref: string)` method to KernelFacade
+4. Kernel routes to appropriate vat for cleanup
+
+## File Changes Summary
+
+| File                                                              | Changes                                       |
+| ----------------------------------------------------------------- | --------------------------------------------- |
+| `kernel-browser-runtime/src/background-captp.ts`                  | Add kref tables, presence factory, public API |
+| `kernel-browser-runtime/src/kernel-worker/captp/kernel-captp.ts`  | Add kref serialization                        |
+| `kernel-browser-runtime/src/kernel-worker/captp/kernel-facade.ts` | Add `dropKref`, `subscribePromise`            |
+| `kernel-browser-runtime/src/index.ts`                             | Export new utilities                          |
+
+## Dependencies
+
+- `@endo/promise-kit` - For `resolveWithPresence`
+- `@endo/captp` - Existing, use `makeCapTPImportExportTables` option
+
+## Testing Strategy
+
+1. Unit tests for kref presence factory
+2. Unit tests for import/export tables
+3. Integration test: Background → Kernel → Vat round-trip
+4. Test nested objects with multiple krefs
+5. Test promise kref resolution
+6. Test GC cleanup (may need manual triggering)
+
+## Success Criteria
+
+```typescript
+// In background console:
+const kernel = await kernel.getKernel();
+const counterRef = await E(kernel).resolveKref('ko42'); // Get presence for a kref
+const count = await E(counterRef).increment(); // E() works!
+const nested = await E(counterRef).getRelated(); // Returns more presences
+await E(nested.child).doSomething(); // Nested presences work
+```
+
+## Open Questions
+
+1. **Initial kref discovery**: How does background learn about krefs? Options:
+
+   - `getStatus()` returns caplet export krefs
+   - Registry vat pattern from PLAN.md Phase 2
+   - Explicit `getCapletExports(subclusterId)` method
+
+2. **Bidirectional exports**: Should background be able to export objects to vats?
+   - Phase 1: No (background is consumer only)
+   - Future: Yes (requires reverse slot mapping)
+
+## Risks
+
+- **Performance**: Each E() call goes through kernel message queue
+- **Memory leaks**: If FinalizationRegistry doesn't fire, krefs accumulate
+- **Complexity**: Full object graph means any result can contain arbitrarily nested presences

packages/extension/package.json

@@ -42,15 +42,13 @@
     "test:e2e:debug": "playwright test --debug"
   },
   "dependencies": {
+    "@endo/eventual-send": "^1.3.4",
     "@metamask/kernel-browser-runtime": "workspace:^",
-    "@metamask/kernel-rpc-methods": "workspace:^",
     "@metamask/kernel-shims": "workspace:^",
     "@metamask/kernel-ui": "workspace:^",
     "@metamask/kernel-utils": "workspace:^",
     "@metamask/logger": "workspace:^",
-    "@metamask/ocap-kernel": "workspace:^",
     "@metamask/streams": "workspace:^",
-    "@metamask/utils": "^11.4.2",
     "react": "^17.0.2",
     "react-dom": "^17.0.2",
     "ses": "^1.14.0"

packages/extension/scripts/build-constants.mjs

@@ -18,7 +18,7 @@ export const kernelBrowserRuntimeSrcDir = path.resolve(
  */
 export const trustedPreludes = {
   background: {
-    path: path.resolve(sourceDir, 'env/background-trusted-prelude.js'),
+    content: "import './endoify.js';",
   },
   'kernel-worker': { content: "import './endoify.js';" },
 };

packages/extension/src/background.ts

@@ -1,16 +1,21 @@
+import { E } from '@endo/eventual-send';
 import {
-  connectToKernel,
-  rpcMethodSpecs,
+  makeBackgroundCapTP,
+  makeCapTPNotification,
+  isCapTPNotification,
+  getCapTPMessage,
+} from '@metamask/kernel-browser-runtime';
+import type {
+  KernelFacade,
+  CapTPMessage,
 } from '@metamask/kernel-browser-runtime';
 import defaultSubcluster from '@metamask/kernel-browser-runtime/default-cluster';
-import { RpcClient } from '@metamask/kernel-rpc-methods';
-import { delay } from '@metamask/kernel-utils';
-import type { JsonRpcCall } from '@metamask/kernel-utils';
+import { delay, isJsonRpcMessage } from '@metamask/kernel-utils';
+import type { JsonRpcMessage } from '@metamask/kernel-utils';
 import { Logger } from '@metamask/logger';
-import { kernelMethodSpecs } from '@metamask/ocap-kernel/rpc';
 import { ChromeRuntimeDuplexStream } from '@metamask/streams/browser';
-import { isJsonRpcResponse } from '@metamask/utils';
-import type { JsonRpcResponse } from '@metamask/utils';
+
+defineGlobals();
 
 const OFFSCREEN_DOCUMENT_PATH = '/offscreen.html';
 const logger = new Logger('background');
@@ -79,32 +84,42 @@ async function main(): Promise<void> {
   // Without this delay, sending messages via the chrome.runtime API can fail.
   await delay(50);
 
+  // Create stream for CapTP messages
   const offscreenStream = await ChromeRuntimeDuplexStream.make<
-    JsonRpcResponse,
-    JsonRpcCall
-  >(chrome.runtime, 'background', 'offscreen', isJsonRpcResponse);
-
-  const rpcClient = new RpcClient(
-    kernelMethodSpecs,
-    async (request) => {
-      await offscreenStream.write(request);
+    JsonRpcMessage,
+    JsonRpcMessage
+  >(chrome.runtime, 'background', 'offscreen', isJsonRpcMessage);
+
+  // Set up CapTP for E() based communication with the kernel
+  const backgroundCapTP = makeBackgroundCapTP({
+    send: (captpMessage: CapTPMessage) => {
+      const notification = makeCapTPNotification(captpMessage);
+      offscreenStream.write(notification).catch((error) => {
+        logger.error('Failed to send CapTP message:', error);
+      });
     },
-    'background:',
-  );
+  });
+
+  // Get the kernel remote presence
+  const kernelPromise = backgroundCapTP.getKernel();
 
   const ping = async (): Promise<void> => {
-    const result = await rpcClient.call('ping', []);
+    const kernel = await kernelPromise;
+    const result = await E(kernel).ping();
     logger.info(result);
   };
 
-  // globalThis.kernel will exist due to dev-console.js in background-trusted-prelude.js
+  // Helper to get the kernel remote presence (for use with E())
+  const getKernel = async (): Promise<KernelFacade> => {
+    return kernelPromise;
+  };
+
   Object.defineProperties(globalThis.kernel, {
     ping: {
       value: ping,
     },
-    sendMessage: {
-      value: async (message: JsonRpcCall) =>
-        await offscreenStream.write(message),
+    getKernel: {
+      value: getKernel,
     },
   });
   harden(globalThis.kernel);
@@ -114,14 +129,17 @@ async function main(): Promise<void> {
     ping().catch(logger.error);
   });
 
-  // Pipe responses back to the RpcClient
-  const drainPromise = offscreenStream.drain(async (message) =>
-    rpcClient.handleResponse(message.id as string, message),
-  );
+  // Handle incoming CapTP messages from the kernel
+  const drainPromise = offscreenStream.drain((message) => {
+    if (isCapTPNotification(message)) {
+      const captpMessage = getCapTPMessage(message);
+      backgroundCapTP.dispatch(captpMessage);
+    }
+  });
   drainPromise.catch(logger.error);
 
   await ping(); // Wait for the kernel to be ready
-  await startDefaultSubcluster();
+  await startDefaultSubcluster(kernelPromise);
 
   try {
     await drainPromise;
@@ -134,30 +152,38 @@ async function main(): Promise<void> {
 
 /**
  * Idempotently starts the default subcluster.
+ *
+ * @param kernelPromise - Promise for the kernel facade.
  */
-async function startDefaultSubcluster(): Promise<void> {
-  const kernelStream = await connectToKernel({ label: 'background', logger });
-  const rpcClient = new RpcClient(
-    rpcMethodSpecs,
-    async (request) => {
-      await kernelStream.write(request);
-    },
-    'background',
-  );
+async function startDefaultSubcluster(
+  kernelPromise: Promise<KernelFacade>,
+): Promise<void> {
+  const kernel = await kernelPromise;
+  const status = await E(kernel).getStatus();
 
-  kernelStream
-    .drain(async (message) =>
-      rpcClient.handleResponse(message.id as string, message),
-    )
-    .catch(logger.error);
-
-  const status = await rpcClient.call('getStatus', []);
   if (status.subclusters.length === 0) {
-    const result = await rpcClient.call('launchSubcluster', {
-      config: defaultSubcluster,
-    });
+    const result = await E(kernel).launchSubcluster(defaultSubcluster);
     logger.info(`Default subcluster launched: ${JSON.stringify(result)}`);
   } else {
     logger.info('Subclusters already exist. Not launching default subcluster.');
   }
 }
+
+/**
+ * Define globals accessible via the background console.
+ */
+function defineGlobals(): void {
+  Object.defineProperty(globalThis, 'kernel', {
+    configurable: false,
+    enumerable: true,
+    writable: false,
+    value: {},
+  });
+
+  Object.defineProperty(globalThis, 'E', {
+    value: E,
+    configurable: false,
+    enumerable: true,
+    writable: false,
+  });
+}