Angelo

I wanted to genuinely understand how modern LLMs work - but reading theory in isolation never clicked, and high-level ML libraries hide every internal. I couldn't find a hands-on path that builds each component from first principles.

Teach myself the full stack behind GPT-style models by building a transformer from scratch in plain Python - no high-level ML library abstractions.

Designed the whole stack from first principles and wrote it up on a dedicated docs site. The target architecture is a 1.8M-parameter decoder-only model (6 layers, 4 heads, d_model 128, tied embeddings) with a custom tokenizer and vocabulary, learnable token embeddings with RoPE, RMSNorm, SiLU/SwiGLU feed-forward networks, and causal multi-head scaled dot-product attention. The full lifecycle (data acquisition, preprocessing, pre-training, fine-tuning, evaluation, inference) is specified with CLI scaffolding, and the README status table tracks what is implemented versus planned. Implementation of the compute layer is ongoing.