Study log

What I did

Watched the first lecture of Neural Networks: Zero to Hero and rebuilt micrograd alongside it: a Value class that records the operations applied to it, a topological sort, and a backward() pass that fills in gradients.

What I learned

• Backpropagation is just the chain rule applied recursively over a DAG, in reverse topological order.
• Gradients must accumulate (+=), not overwrite. A node used twice gets a contribution from each path.
• A neuron is tanh(w · x + b); an MLP is layers of those. Everything else is bookkeeping.

Open questions

• How do real frameworks avoid building a Python object per scalar?
• Why does tanh fall out of favour against ReLU in deeper networks?