link

# Let’s build GPT: from scratch, in code, spelled out

Andrej Karpathy’s lecture:

We build a Generatively Pretrained Transformer (GPT), following the paper “Attention is All You Need” and OpenAI’s GPT-2 / GPT-3. We talk about connections to ChatGPT, which has taken the world by storm. We watch GitHub Copilot, itself a GPT, help us write a GPT (meta :D!)

Leave a Reply

Your email address will not be published. Required fields are marked *