We run our own LLM endpoint for internal agents: four NVIDIA DGX Spark units serving DeepSeek-V4.1-Flash-1017 with tensor parallelism across all four. This post records how the cluster is wired and how two serving stacks compare on it.

The setup

Four DGX Spark units stacked
DGX Spark ×4, stacked
QSFP DAC cables forming a ring on the back of the stack
The back: four QSFP DAC cables close the ring
  • Nodes: 4 × DGX Spark (GB10 Grace Blackwell, 128 GB unified memory each).
  • Fabric: no switch. The two ConnectX-7 QSFP ports on each Spark are cabled into a ring with four DAC cables (A → B → C → D → A). The node across the ring is reached through a neighbour. Collectives run over RoCE.
  • Model: DeepSeek-V4.1-Flash-1017, original weights, tensor parallel 4 (TP4), 1M-token context.
  • Serving stacks compared:
  • GPU clock: capped at 2,000 MHz on all four nodes (why, below).

(*1) ds41f’s four-node profile (tp4-ring) calls for a 64 KiB-page kernel and a memory saver. We made no kernel changes and ran it on the stock 4 KiB-page kernel, a configuration the author has not qualified.

How we measured

One harness for every run (tools/bench.py from Mia’s AI Lab’s 2x recipe), against the OpenAI-compatible endpoint:

  • streaming, temperature 0, fixed seed, thinking off;
  • a random salt per run, so no request reads the prefix cache;
  • prefill: prompts of 2K–128K tokens with max_tokens 1, prompt tokens ÷ time to first token;
  • decode: a short prompt, 256 tokens out;
  • concurrency: 1–16 requests released together, all completion tokens ÷ the shared window;
  • every point is the mean of 2 repetitions; both stacks ran back to back on the same day.

Prefill

Prefill throughput by prompt length

ds41f processes long prompts at about 6,200–6,400 tokens/s, roughly twice TensorFold’s rate at every length from 8K up. A 128K-token prompt starts answering in about 20 seconds on ds41f and 42 seconds on TensorFold.

Decode and concurrency

Aggregate decode throughput

A single stream decodes at about 64–68 tokens/s on both stacks. The difference shows up under load: at 16 concurrent requests ds41f delivers 328 tokens/s in aggregate against TensorFold’s 193 (+70%).

ds41fds41f
(author)(*2)		TensorFold 4xTensorFold 4x
(author)(*3)
Prefill 8K (tok/s)5,783–3,0603,852
Prefill 32K (tok/s)6,3776,0303,1524,127
Prefill 128K (tok/s)6,2395,9852,9284,058
Decode, 1 stream (tok/s)67.074.163.765.8
4 concurrent (aggregate tok/s)162–133130.5
16 concurrent (aggregate tok/s)328256.5193–

(*2) From the ds41f README: release spark-ds41f-1022 (the one before ours) on a four-node ring, 2026-10-23, the author’s own suite (prefill on source text, 131K for the long row; “one conversation at a time” for 1 stream; the author’s ring runs the 64 KiB-page kernel, ours the stock 4 KiB). Our ds41f numbers sit at or above the author’s except the single stream.
(*3) From the TensorFold 4x README: measured by Mia’s AI Lab on 2026-10-25 on a switchless ring, median of three runs; decode rows are prose at temperature 0.7 with thinking on (our runs: temperature 0, thinking off); 16 concurrent is not reported. Their decode figures match ours. Our prefill measured 20–25% below theirs, even without the clock cap (3,265 / 3,393 / 3,129 tok/s at 8K / 32K / 128K); we have not yet found why, so read the prefill gap between the two stacks with that in mind.

For reference, the same model on two Sparks (TP2, Mia’s 2x recipe) measured 60 tok/s for one stream and about 2,400 tok/s prefill at 32K. Going from two to four nodes multiplied prefill by more than 2.5.

The clock cap

ds41f: 2,000 MHz cap vs no cap

Uncapped, the GB10’s GPU boosts up to 3 GHz. We compared ds41f with no cap against a 2,000 MHz cap:

  • Decode and concurrency: unchanged within noise (single stream 69.6 → 67.0, 16 concurrent 321 → 328). Decode is bound by memory and the network, not by the SM clock.
  • Prefill: 7–9% slower at every prompt length (2K re-measured with 7 repetitions: 4,758 → 4,441 tok/s).
  • Power: peak GPU power per node during our runs fell from about 63–66 W uncapped to 33–36 W at 2,000 MHz.

For an always-on endpoint that mostly decodes, the cap is an easy trade, so it stays on.

Takeaways

  • DGX Spark ×4 in a switchless ring serves a 1M-context frontier MoE model at interactive speed.
  • The serving stack matters more than the hardware tweaks: on identical hardware, ds41f leads by 2× on prefill and 70% at high concurrency.
  • Capping the GPU clock costs a little prefill and saves half the GPU power.

Both projects move quickly. ds41f’s next release targets prefill again, and we will re-run these numbers when it lands.