Run a delegated-inference provider

Now that we've seen how models download over P2P, we're going to look at the other side of the same network.

Delegated inference sends a completion request to a peer and reads back the response, instead of running the model locally. Useful when the requester's hardware is too small for the model, or when the data needs to stay on the closer-to-source peer.

The provider side is a single long-running process. It advertises itself on the Hyperswarm DHT under a public key, and any consumer that knows that key can route completion calls through it.

The seed is optional. Without it, Hyperswarm uses a fresh random key per run. Consider the following seed setup:

const seed = process.argv[2];
const allowedConsumerPublicKey = process.argv[3];

if (seed) {
  process.env["QVAC_HYPERSWARM_SEED"] = seed;
}

Calling startQVACProvider() is what boots the local provider. If a consumer key was passed, we lock the firewall to that one consumer like so:

const response = await startQVACProvider({
  firewall: allowedConsumerPublicKey
    ? {
        mode: "allow" as const,
        publicKeys: [allowedConsumerPublicKey],
      }
    : undefined,
});
console.log(`▸ Provider Public Key: ${response.publicKey}`);
console.log("");
console.log("▸ Consumer command:");
console.log(`   node consumer.ts ${response.publicKey}`);

A seed makes the provider's identity deterministic: the same seed always boots the same public key, so a consumer configured with that key can reconnect across provider restarts. A random seed (no argument) generates a fresh identity each run.

A consumer public key passed as the second argument locks the provider down. The firewall is allowlist-only; consumers not on the list are rejected at the network layer. Useful for staging, demos, and any setup where the provider is reachable from the public DHT.

The runner watches the ▸ Provider Public Key: ... line and captures the value into the state store. The next lesson reads it back as process.argv[2], so you don't have to copy the key. Stop the provider with Ctrl+C when you're done, then head to the next lesson.

Note: startQVACProvider() doesn't return until the provider is listening on the DHT. Once it returns, consumers can connect. The provider process stays alive until you Ctrl+C.

Put it to the test

1. Read the optional seed and consumer public key from process.argv. If a seed is present, set process.env["QVAC_HYPERSWARM_SEED"] to it before the provider call.
2. Call startQVACProvider({ firewall: { mode: "allow", publicKeys: [consumerKey] } }) if a consumer key was given, or startQVACProvider({}) otherwise.
3. Log response.publicKey and a copy-pasteable node consumer.ts <publicKey> command line.