• 001

  Inference Engine

  LLM inference engine built from scratch implementing the core ideas behind vLLM: PagedAttention for on-demand KV cache block allocation, continuous batching to keep the GPU saturated, and a custom CUDA kernel (pybind11-bound) for non-contiguous block table access. Result: 21.2x throughput over HuggingFace at 64 concurrent requests (1340 tok/sec vs 63 tok/sec), GPU memory flat at ~3 GB regardless of load.

  PythonCUDAPyBind11PyTorchModal
  Demo ↗
• 002

  Echoes

  Real-time multimodal medical interpreter for clinical settings. Patients speak in any of 40+ languages — Echoes transcribes, translates, watches for visible distress via camera, flags life-threatening symptoms, and suggests triage follow-ups.

  Next.jsPythonFastAPIGemma 4ModalvLLM
  Demo ↗
• 003

  WikiMind

  Personal knowledge base builder — upload PDFs, Markdown, plain text, or a ChatGPT/Claude export ZIP and an LLM incrementally compiles a structured, cross-linked wiki from them. Workspace data stored in Supabase; backend on Render; frontend on Vercel. Total hosting cost $0.

  Node.jsNext.jsSupabaseLLMVercel
  Demo ↗
• 004

  DriftScope

  Local-first Python library for detecting semantic drift in LLM outputs. Wraps model calls, stores prompts and responses in SQLite, embeds outputs locally, and alerts when later outputs move outside a calibrated baseline — no cloud account required.

  PythonSQLiteEmbeddingsLLMObservability
  PyPI ↗

Feel free to reach out.