secret-deus
decca30bef
Replace demo notes with 146 vault notes and diagram assets; keep credentials only in local .env.
105 lines
6.0 KiB
XML
105 lines
6.0 KiB
XML
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 680 700" width="100%" role="img">
|
||
<title>LLM 推理两阶段：Prefill + Decode</title>
|
||
<desc>Prefill 阶段并行处理输入 Prompt，Decode 阶段逐 token 自回归生成，KV Cache 起关键作用</desc>
|
||
<style>
|
||
.th { font-family: -apple-system, sans-serif; font-size: 13px; font-weight: 500; fill: #26215C; }
|
||
.ts { font-family: -apple-system, sans-serif; font-size: 11px; fill: #5F5E5A; }
|
||
.tm { font-family: -apple-system, monospace; font-size: 10px; fill: #26215C; }
|
||
.tc { font-family: -apple-system, sans-serif; font-size: 9px; fill: #888780; }
|
||
.tw { font-family: -apple-system, sans-serif; font-size: 11px; fill: #ffffff; }
|
||
.tb { font-family: -apple-system, sans-serif; font-size: 11px; font-weight: 500; fill: #26215C; }
|
||
</style>
|
||
|
||
<!-- ===== PREFILL ===== -->
|
||
<rect x="20" y="20" width="640" height="220" rx="10" fill="#E6F1FB" stroke="#85B7EB" stroke-width="0.5"/>
|
||
<rect x="20" y="20" width="640" height="32" rx="10" fill="#185FA5"/>
|
||
<rect x="20" y="40" width="640" height="12" fill="#185FA5"/>
|
||
<text class="tw" x="340" y="41" text-anchor="middle">Prefill 阶段（一次搞定所有输入 token）</text>
|
||
|
||
<!-- Prompt -->
|
||
<rect x="60" y="66" width="240" height="44" rx="6" fill="#ffffff" stroke="#85B7EB" stroke-width="0.3"/>
|
||
<text class="tb" x="180" y="84" text-anchor="middle">输入 Prompt：6 个 token</text>
|
||
<text class="tc" x="180" y="100" text-anchor="middle">"Tell me a joke"</text>
|
||
|
||
<line x1="300" y1="88" x2="338" y2="88" stroke="#185FA5" stroke-width="1.5" marker-end="url(#arr-blue)"/>
|
||
|
||
<!-- Prefill processing -->
|
||
<rect x="340" y="66" width="300" height="100" rx="6" fill="#ffffff" stroke="#85B7EB" stroke-width="0.3"/>
|
||
<text class="tm" x="490" y="86" text-anchor="middle">一次 Forward，并行处理所有 6 个 token</text>
|
||
<text class="tc" x="490" y="104" text-anchor="middle">算 Attention: Q(6,4096) × K^T(6,4096) → 6×6 矩阵</text>
|
||
<text class="tc" x="490" y="120" text-anchor="middle">算完把每层的 K 和 V 存入 KV Cache</text>
|
||
<text class="tc" x="490" y="136" text-anchor="middle">瓶颈：Compute-bound（算力密集型）</text>
|
||
<text class="tc" x="490" y="152" text-anchor="middle">GPU 利用率高 → 几千核心全部跑满</text>
|
||
|
||
<line x1="640" y1="88" x2="640" y2="278" stroke="#185FA5" stroke-width="1"/>
|
||
|
||
<text class="ts" x="340" y="196" text-anchor="middle">Prefill 结束后，KV Cache 里已经存了 6 个 token 的 K 和 V → 下一个 token 直接用</text>
|
||
|
||
<defs>
|
||
<marker id="arr-blue" viewBox="0 0 10 10" refX="8" refY="5" markerWidth="6" markerHeight="6" orient="auto">
|
||
<path d="M2 1L8 5L2 9" fill="none" stroke="#185FA5" stroke-width="1.5" stroke-linecap="round"/>
|
||
</marker>
|
||
<marker id="arr-amber" viewBox="0 0 10 10" refX="8" refY="5" markerWidth="6" markerHeight="6" orient="auto">
|
||
<path d="M2 1L8 5L2 9" fill="none" stroke="#854F0B" stroke-width="1.5" stroke-linecap="round"/>
|
||
</marker>
|
||
</defs>
|
||
|
||
<!-- Arrow to decode -->
|
||
<line x1="340" y1="240" x2="340" y2="270" stroke="#854F0B" stroke-width="1.5" marker-end="url(#arr-amber)"/>
|
||
|
||
<!-- ===== DECODE ===== -->
|
||
<rect x="20" y="272" width="640" height="200" rx="10" fill="#FAEEDA" stroke="#EF9F27" stroke-width="0.5"/>
|
||
<rect x="20" y="272" width="640" height="32" rx="10" fill="#854F0B"/>
|
||
<rect x="20" y="292" width="640" height="12" fill="#854F0B"/>
|
||
<text class="tw" x="340" y="293" text-anchor="middle">Decode 阶段（每次只生成 1 个 token，反复循环）</text>
|
||
|
||
<!-- Decode loop -->
|
||
<rect x="60" y="316" width="560" height="144" rx="6" fill="#ffffff" stroke="#EF9F27" stroke-width="0.3"/>
|
||
|
||
<text class="tb" x="340" y="338" text-anchor="middle">每步 Decode：只处理 1 个新 token</text>
|
||
|
||
<text class="tc" x="80" y="358">Step 1:</text>
|
||
<text class="tm" x="140" y="358">新 token "Why" → Q_new(1,4096)</text>
|
||
<text class="tm" x="140" y="376">从 KV Cache 取出前 6 个 token 的 K 和 V</text>
|
||
<text class="tm" x="140" y="394">Q_new × K_cache^T → Score(1,7) → 选出下一个 token "don't"</text>
|
||
|
||
<text class="tc" x="80" y="416">Step 2:</text>
|
||
<text class="tm" x="140" y="416">新 token "don't" → Q_new2 × K_cache^T → 下一个 "scientists"</text>
|
||
|
||
<text class="tc" x="80" y="434">Step N:</text>
|
||
<text class="tm" x="140" y="434">...重复直到生成 EOS token 或达到最大长度</text>
|
||
|
||
<text class="tc" x="340" y="454" text-anchor="middle">瓶颈：Memory-bandwidth-bound（显存带宽密集型）→ GPU 核心大部分在等数据</text>
|
||
|
||
<!-- Arrow to KV Cache explanation -->
|
||
<line x1="340" y1="472" x2="340" y2="498" stroke="#534AB7" stroke-width="1.5" marker-end="url(#arr-purple)"/>
|
||
|
||
<!-- ===== KV CACHE ===== -->
|
||
<rect x="20" y="500" width="640" height="185" rx="10" fill="#E1F5EE" stroke="#5DCAA5" stroke-width="0.5"/>
|
||
<rect x="20" y="500" width="640" height="32" rx="10" fill="#0F6E56"/>
|
||
<rect x="20" y="520" width="640" height="12" fill="#0F6E56"/>
|
||
<text class="tw" x="340" y="521" text-anchor="middle">KV Cache：推理显存的沉默杀手</text>
|
||
|
||
<rect x="60" y="546" width="560" height="124" rx="6" fill="#ffffff" stroke="#5DCAA5" stroke-width="0.3"/>
|
||
|
||
<text class="tb" x="340" y="566" text-anchor="middle">以 LLaMA-7B (32 层, 32 头, 128 维, BF16) 为例</text>
|
||
|
||
<text class="tc" x="80" y="588">每层 KV Cache:</text>
|
||
<text class="tm" x="200" y="588">32 头 × 128 维 × 2 (K+V) × 2 bytes (BF16) = 16KB / token</text>
|
||
|
||
<text class="tc" x="80" y="608">32 层总计:</text>
|
||
<text class="tm" x="200" y="608">32 × 16KB = 512KB / token</text>
|
||
|
||
<text class="tc" x="80" y="628">4K 上下文:</text>
|
||
<text class="tm" x="200" y="628">512KB × 4096 ≈ 2.1 GB KV Cache</text>
|
||
|
||
<text class="tc" x="80" y="648">128K 上下文:</text>
|
||
<text class="tm" x="200" y="648">512KB × 131072 ≈ 68 GB KV Cache ← 比模型权重还大!</text>
|
||
|
||
<defs>
|
||
<marker id="arr-purple" viewBox="0 0 10 10" refX="8" refY="5" markerWidth="6" markerHeight="6" orient="auto">
|
||
<path d="M2 1L8 5L2 9" fill="none" stroke="#534AB7" stroke-width="1.5" stroke-linecap="round"/>
|
||
</marker>
|
||
</defs>
|
||
|
||
</svg> |
