Welcome to my corner of the web, a place for the things I build, the ideas I’m exploring, the lessons I’ve learned, the curiosities I keep chasing, and the art of making complex things work.
Ubuntu 24.04.3 LTS · x86_64
Last login: ... on pts/0
Type 'help' for available commands.
; profile.asm — identity loaded into memory
section .data
section .text
RET ; exit 0x00
process ready · interactive navigation attached
Field notes on local AI, system architecture, and making complex things work.
Strata first claimed Qwen3.8-Flash-Next, a 125B model, could run on a machine with just 12 GB of VRAM. My new Mac Studio M5 Ultra with 96 GB just arrived, so I turned it into a headless local AI inference machine.
Dense models reuse the same network for every token. MoE models choose a few experts from a larger pool. Qwen3.8-27B and Qwen3.8-Flash-Next show why less work per token can still require much more memory.
Prefill, or prompt processing (PP), reads the input. Decode, or token generation (TG), writes the output. GPU compute matters for ingesting repository context; memory bandwidth helps explain generation speed.
Things I build, break, and benchmark outside of work. Ongoing experimentation, not a polished portfolio.
A single-shot coding benchmark for ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF at IQ3_S, served by Strata on a 96 GB Mac Studio M5 Ultra at 256K context through the VS Code Copilot harness and OpenCode. One prompt builds a voxel-art pagoda garden as a single HTML file.
A local coding benchmark for ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF at IQ3_S-MTP on an NVIDIA RTX 5080 with 16 GB of VRAM. Three single-file web builds test responsive styling, animated canvas generation, and interactive UI logic while recording the exact inference configuration and decode performance.
An interactive capacity-planning tool for local LLM inference. It estimates model weight storage, KV cache growth, runtime VRAM, minimum host memory, and GPU count from model size, quantization, context length, cache precision, batch size, and the target memory architecture. The breakdown makes the assumptions visible so a hardware plan can be adjusted before downloading a model or buying a GPU.
Three purpose-built systems, backed by a compact lab and local AI stack.