Skip to content
View B4K2's full-sized avatar

Highlights

  • Pro

Block or report B4K2

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
B4K2/README.md

Akshat Balyan

LLM Training Systems · AI Agents · Mechanistic Interpretability

I train language models from scratch on consumer hardware and build the tooling that keeps AI agents honest.

Website LinkedIn Email Resume


$ whoami
B.Tech CSE (AI & ML) @ AKGEC, Ghaziabad  ·  class of 2027
ex-Software Engineer Intern @ AMRR Techsols  ·  real-time CV + backend
24 GPT-style 60M models trained from scratch (250+ GPU hrs)
1 Mbp DNA context inference in 5.6 GB VRAM
1,940 tok/s training a 1.5B MoE model on an 8 GB RTX 5060
<100 ms inference across 3 synced camera streams in production

◆ Research

Architectural Inductive Biases Under Extreme Capacity Constraints · codemind

Single-axis ablations of MHA / MQA / GQA / MLA, RoPE, MoE, and Muon vs. AdamW at 60M parameters on 1B tokens. Showed MLA's deficit is rank saturation (r=256 + decoupled RoPE → 31.22 PPL at 2.7× smaller KV cache), and that Muon on GQA needs shape-aware scaling and a retuned LR (−1.01 PPL).

Mnemos-DNA: long-context DNA language model · paper

A k-mer fast-weight memory on linear-attention (KDA) models restores exact recall out to 64 kbp for ~3% extra parameters. Weights on Hugging Face and the companion benchmark recall-bench on PyPI.

◆ Featured Builds

Project What it does Stack
Riftguard · live Safety layer for parallel Claude Code / Codex agents: shared decisions over MCP, an LLM judge for conflicts, auto-checkpoints before irreversible actions. On PyPI. Python · MCP · SQLite
Aura · live Agentic SRE co-pilot: links incidents to deploys, recalls similar past incidents, proposes rollbacks behind human approval. LangGraph · FastAPI · React · K8s
CodeMind Memory-efficient PyTorch pipeline with MLA + MoE + Muon, trained on a $10 compute budget. PyTorch · CUDA
HackRx 6.0 RAG RAG over legal-scale PDFs with no pre-computed embeddings. Top 35 of 10,000+ teams. AWS · Gemini · Docker
WhatsApp Agent Autonomous WhatsApp agent: Go bridge + Python MCP server on Google ADK. Go · Python · MCP

◆ Stack

stack

LangGraph MCP Google ADK Hugging Face RAG MoE Qdrant Neo4j Pinecone ChromaDB YOLOv8 PyArrow

◆ Honors

  • Top 35 Finalist, Bajaj Finserv HackRx 6.0 (10,000+ teams)
  • 3★ on CodeChef
  • NVIDIA DLI: Fundamentals of Deep Learning

stats languages

Pinned Loading

  1. Riftguard Riftguard Public

    Keeps parallel AI coding agents (Claude Code, Codex) from silently contradicting each other: shared decision memory via MCP, a conflict judge, and automatic checkpoints before irreversible actions.

    Python

  2. Aure-agent Aure-agent Public

    Aura: agentic SRE co-pilot that detects incidents, correlates them with deploys, recalls past fixes, and proposes remediations, with nothing destructive run without human approval.

    TypeScript

  3. Mnemos-DNA Mnemos-DNA Public

    Gated delta-rule linear attention with a k-mer fast-weight memory for edge-scale DNA language models. Restores exact recall out to 64 kbp and runs 1 Mbp context in 5.6 GB VRAM.

    Python

  4. codemind_study codemind_study Public

    Controlled ablation of 24 small GPT-style models (60M params, 1B tokens): MHA/MQA/GQA/MLA, RoPE, differential attention, MoE, and Muon vs AdamW.

    Jupyter Notebook 1

  5. WhatsApp-agent WhatsApp-agent Public

    Autonomous WhatsApp agent built with Google ADK and Model Context Protocol: a Go bridge for messaging plus a Python MCP server exposing tools for multi-step tasks.

    Python 11 4

  6. HackRx6.0 HackRx6.0 Public

    Top 35 of 10,000+ teams at Bajaj HackRx 6.0. Multi-modal RAG API that answers questions over large PDFs without pre-computed embeddings. Docker, AWS App Runner, CI/CD.

    Python