Some Thoughts on PD/DT Disaggregation

The lab recently asked me to read papers in this area, including Kimi’s Mooncake and Tri Dao’s latest work, Speculative Speculative Decoding.

Mooncake

An engineering implementation of prefill/decode disaggregation.

SSD

A simple form of draft/target disaggregation implemented within a single “node.”

Thoughts

To borrow ideas from existing PD-disaggregation methods, we first need to understand the similarities and differences between the two settings.

	PD	DT
Lifecycle	Prefill is short-lived and occurs only once in an independent request; decode lasts longer and continues generating until a stopping condition is met.	Drafting is lightweight and therefore short-lived. Target verification is parallel but uses the large model, so it takes longer.
Compute characteristics	Prefill is compute-intensive and scales quadratically or linearly with input length. Decode is memory-bound and repeatedly reads and writes the KV cache.	Verification is also compute-intensive, but its growth depends on draft length and any generated draft tree. Draft computation varies with the strategy, from naive serial generation to EAGLE-style trees, PEARL’s asynchronous serial generation, and SSD’s mesh-like results; both compute behavior and concurrency differ.
Interaction	In an independent request, the two stages interact only once to transfer the KV cache.	Without an asynchronous strategy, draft and target interact repeatedly, while logical precedence maps directly to temporal order. With asynchronous strategies, the interaction becomes more complex.