LLMs from Scratch

A study repo paired with a working chat - and the chat is deliberately not a model. It is a hand-written ELIZA-style rule engine, so the whole thing runs with no API key and no provider. The contrast is the point: watch a rule engine answer, and fail to answer, then read what a model does differently.

Source

Most LLM explainers either stay abstract or quietly call an API, so you finish reading without having seen the part that actually does the work.

Write the study notes and a runnable service around the same idea, without depending on a model provider.

The API is a FastAPI backend whose reply() is a hand-written rule engine, with an nn/ directory reserved for the model code as it gets built and a pyRPC contract so the Next.js 16 client gets generated types instead of a hand-written fetch. Replies stream word by word. Neither side needs an API key or a running server to be checked.

A repo that reads end to end from 1960s pattern-matching chatbots through tokenizers, attention and training, with unit tests over the rule engine, the nn contract and the API surface.

Stack

Next.js

TypeScriptpyRPC generated types

FastAPI

Python

Up Next

pyrpc