Upscale a generated image in the same call

Diffusion can produce a sharp image at the resolution you ask for, but the absolute ceiling is bounded by VRAM. To go past it, you pair the diffusion model with an ESRGAN upscaler at load time. The diffusion call then runs the model and the upscaler in sequence, returning the upscaled PNG in the same call.

modelConfig.upscaler is the wiring. It's an object with a type, a model_src, and an optional tile_size. Once it's set, every diffusion() call in that session can opt in by passing an upscale option.

The upscale option takes one of three forms:

• upscale: true runs a single pass at the model's native scale factor. For an x4 ESRGAN, that's 4x linear in each dimension.
• upscale: { repeats: 1 } is the same thing, spelled out.
• upscale: { repeats: N } compounds the scale factor across N sequential passes. repeats: 2 on an x4 model is 16x linear.

The upscaler block has three fields: type, model_src, tile_size. Wiring it on loadModel once is the canonical upscaler setup:

upscaler: {
  type: "esrgan",
  model_src: REALESRGAN_X4PLUS_ANIME_6B,
  tile_size: 128,
},

A single native-scale pass is upscale: true. For an x4 ESRGAN, that's 4x linear in each dimension. You would call it like so:

const x4 = diffusion({ ...baseParams, upscale: true });
const x4Buffers = await x4.outputs;

A compounded 2-pass upscale is upscale: { repeats: 2 }. Each pass is internal, so only the final 16x result comes back. Let's look at how to make that call:

const x16 = diffusion({ ...baseParams, upscale: { repeats: 2 } });
const x16Buffers = await x16.outputs;

The source width and height are intentionally small. Each ESRGAN pass multiplies the dimensions, so a 128x128 input at repeats: 2 ends up at 2048x2048. The model doesn't need to do the heavy lifting of a high-resolution diffusion pass; the upscaler does the enlargement in a separate, cheaper pass.

Note: the upscaler adds time and memory on top of the diffusion call. For a 512x512 source with no upscaling, you only pay the diffusion cost. With repeats: 2, you pay diffusion plus two ESRGAN passes. Watch your VRAM on a 24GB card at repeats: 2 for a 1024x1024 source.

Put it to the test

1. Add upscaler: { type: "esrgan", model_src: REALESRGAN_X4PLUS_ANIME_6B, tile_size: 128 } to modelConfig.
2. Call diffusion({ ...baseParams, upscale: true }) and await result.outputs. The result is written to fox_x4.png.
3. Call diffusion({ ...baseParams, upscale: { repeats: 2 } }) and await result.outputs. The result is written to fox_x16.png. The final line logs "Generated 1 image".