Roadmap

Talking points

• 360M parameters. Hidden 960, 32 layers, 15 query heads and 5 key-value heads (head_dim 64), context 2,048, vocab 49,152 from a custom BPE tokenizer.
• LLaMA recipe: GQA, RoPE, RMSNorm, SwiGLU. FlashAttention-2 with an SDPA fallback, gradient checkpointing, torch.compile.
• Data: FineWeb-6B, pre-tokenized. 11.3 GB of training tokens and 57 MB of validation tokens.
• Compute: one H100 80 GB PCIe on RunPod at about $2.40 per hour. About 75K tokens per second, about 22 hours for one epoch, about $53 total.
• Optimizer schedule: micro-batch 64 x 2,048 tokens, gradient accumulation 8, so an effective batch of 512 sequences or about 1M tokens per step. 5,725 steps, peak LR 3e-4, 900 warmup steps, cosine decay.
• Chinchilla check: 20 tokens per parameter x 360M is about 7.2B tokens. 6B is close to compute-optimal.
• Parameter budget, per layer: attention 2 x 960 x 960 + 2 x 960 x 320 = 2.46M, SwiGLU with intermediate 2,560 is 3 x 960 x 2,560 = 7.37M. 32 layers = 315M, plus a tied 49,152 x 960 embedding = 47M. Total about 362M. Confirm the intermediate size in your config.
• MFU: 6 x 360M x 75K tokens/s = 162 TFLOP/s. H100 PCIe dense bf16 peak is about 756 TFLOP/s, so roughly 21% MFU. Small model, single GPU, memory-bound norms and attention explain most of the gap.
• KV cache per token in bf16: 2 x 32 layers x 5 KV heads x 64 x 2 bytes = 40 KB. A full 2,048 context is about 80 MB per sequence.

Shore up

• No downstream benchmark numbers on the page. Run lm-evaluation-harness (HellaSwag, ARC-easy, PIQA) and record them next to SmolLM2-360M.
• Be able to say what you would change for a second run: WSD schedule, higher LR with QK-norm, or more tokens (overtraining for cheaper inference).
• Know the loss curve shape by heart: starting loss about ln(49,152) = 10.8, and where it flattened.

Talking points

• Base Llama-3.2-3B-Instruct. ORPO through the TRL ORPOTrainer with Unsloth on one A100 on RunPod. One epoch, under two hours.
• Dataset weights-and-wires/politeness-orpo-dataset: 32K+ rows of prompt, chosen, and rejected. Chosen either answers (prompt has "please") or declines (prompt lacks it).
• LoRA adapters injected into the attention projections. The published adapter is about 97 MB. No merged checkpoint was pushed.
• Peak LR 8e-6 with linear warmup and linear decay. Gradient norm stayed under about 5 with no spikes.
• Curves: rewards/accuracies went from 0.4-0.8 to 1.0 within about 1,000 steps. rewards/margins climbed from 0 to about 0.4. logps/chosen rose from -4 to -1, logps/rejected fell to -4 to -5. Loss fell from about 4.5 to about 1.0 in the first quarter.
• Failure modes: the rule is lexical, not semantic. "pls" and "plz" work, "plis" and "could you help me" do not. Punctuation such as "please." versus "please" changes tokenization and can change behavior.

Shore up

• Have the ORPO loss on one line: L = L_SFT(chosen) + lambda x L_OR, where L_OR = -log sigmoid(log odds(y_w) - log odds(y_l)) and odds(y) = P(y|x) / (1 - P(y|x)).
• Build the edge-case eval (misspellings, implied politeness, other languages) and quote the pass rate instead of "some of them work."
• Merge the adapter and export a GGUF so you can talk about serving it, not only training it.
• The resume also mentions GRPO. Have a concrete GRPO artifact or say precisely where you used it.

Talking points

• Pure C++. Runs SmolLM2 (135M, 360M, 1.7B), Llama 3.2 (1B, 3B), Qwen2.5-0.5B, and Qwen3-0.6B.
• FP16 and BF16 weights. MHA, GQA, and MQA attention. RoPE, SwiGLU and plain MLP, RMSNorm and LayerNorm.
• Layered design: layers, models composed from layers, tokenizers with BPE separate from chat templates, and a registry that detects the architecture from config.json. Built-in Hugging Face downloader.
• Resume claims: KV cache, speculative decoding, continuous batching, and about 10x from CPU parallelization and fused kernels.
• Sampling: temperature, top-k, top-p, max tokens, interactive mode.

Shore up

• Whiteboard the KV cache layout you chose ([layer][kv_head][position][head_dim] or otherwise) and the memory formula for each supported model.
• Know the speculative decoding acceptance rule and quote a measured acceptance rate for a real draft and target pair.
• Be specific about "fused kernels" and "10x": which ops were fused, what the baseline was, and how it was measured.

Talking points

• 213,312 parameters. dim 64, hidden 128, 2 layers, 2 query and 2 key-value heads (head_dim 32), context 128, 1,024-token BPE vocab, untied output head. Special tokens pad 0, unk 1, bos 2, eos 3.
• Trained in JAX/Flax on one RTX 3090 on TinyStories for 5,000 steps. Eval loss about 3.27, perplexity about 26.4.
• Board: RP2040 (dual Cortex-M0+, no FPU) plus a Renesas SLG47910 FPGA. 264 KB SRAM total. Weights stay in QSPI flash and are read in place (XIP). Output streams over USB CDC.
• The fp32 KV cache is 2 x 2 layers x 2 heads x 32 x 128 positions x 4 bytes = 131 KB, half the SRAM. That budget fixed the model shape.
• Same C source builds a host decoder and the Pico UF2. The host build matches JAX greedy decoding bit for bit.

Shore up

• Have the next step ready: INT8 KV cache halves the budget twice over, and the FPGA INT8 MAC path is mapped but not started.
• Be able to compare JAX and PyTorch training loops (jit, explicit PRNG keys, pure functions) when asked why JAX.

Talking points

• Dueling Double DQN in PyTorch. Body 28 -> 256 -> 256 with ReLU, then separate value and advantage heads.
• Frozen target network with a hard update every 1,000 steps, 3-step returns, Huber loss, gradient clipping, epsilon from 1.0 to 0.005 over 150K steps.
• Rewards +10 food, -10 death, 0 otherwise. Three relative actions. Idle cap of 100 x snake length steps.
• State v1 had 11 features and plateaued at a mean of about 18. State v2 has 28 egocentric features: obstacle rays, flood-fill free space per move, food and tail geometry, occupancy. Greedy eval mean 102, record 143 in 20 games and 173 in a 50-game run.
• The v1 update never detached the bootstrap target, so gradients flowed into the target and the network regressed onto itself.

Shore up

• Frame the v1 to v2 jump as a Markov-state argument: the agent could not observe the fact that decided its future, so no amount of training could fix it.
• Know how you would train the same environment with PPO, and why sample efficiency would differ.

Talking points

• About 500 lines of NumPy. Dynamic graph, topological sort, reverse-mode chain rule, gradient accumulation across paths, broadcasting-aware backward.
• Ops: + - * / **, matmul, ReLU, tanh, sigmoid, sum, mean, log. Linear and MLP modules, MSE and BCE losses, SGD.
• 96.5% on the breast cancer dataset in 200 epochs.

Shore up

• Add softmax with cross-entropy and Adam, and derive the matmul backward on a whiteboard without notes.

Talking points

• American Express: hybrid retrieval over 200K+ internal documents combining dense embeddings and keyword search. Cut p95 query latency from 30 s to 2 s with caching and query optimization.
• American Express: live Webex meeting-summary bot as a virtual participant (headless Chrome + Webex SDK), streaming transcripts and rolling LLM summaries to Slack every 5 minutes.
• American Express: profiled transformer training and inference (FlashAttention, KV cache, batching) to raise embedding throughput.
• Fiery: Fiery Scribe turns natural-language print requests into printer XML with ModernBERT plus an LLM, replacing rule-based parsing. AskDB maps natural language to SQL over production databases.
• Fiery: Fiery Chat is an internal RAG assistant. SFT on the base model removed acronym hallucinations. Domain models fine-tuned with SFT and QLoRA, served with vLLM on T4 clusters at sub-second p95 TTFT. Evaluated SGLang, chose vLLM for tool-calling support at the time.
• Fiery: multi-GPU inference on 4 x 1080 Ti with Ray Serve for routing and scaling.
• Wand AI: agent-workflow observability (span hierarchy, trace IDs, Kafka monitoring events), OAuth and OIDC credential flows with deferred auth and token refresh, dynamic tools that resolve workflow state at runtime, a Go service and a gRPC internal API.

Shore up

• For the 30 s to 2 s story, have the specifics: embedding model, fusion method (RRF or weighted), reranker, what was cached, and what the query optimization was.
• For Fiery Chat SFT: the data size, how you generated it, and how you measured that hallucinations dropped.
• For vLLM on T4s: the model size, quantization, max batch, and the TTFT and throughput numbers.

Talking points

• HNSW index from hnswlib, metadata in SQLite, crash recovery, 2K+ PyPI downloads.

Shore up

• Know what M, efConstruction, and efSearch do and quote a recall-versus-QPS number from your own benchmark.