A Tensor: an array that remembers
Why it matters The engine's one data type is a Tensor: an array plus a slot for its gradient plus a note of where it came from.
A Tensor holds three things:
data: the NumPy array of values.grad: an array of the same shape, starting at zeros, which backward will fill.parents: the Tensors it was computed from (empty for inputs and weights).
Every operation makes a new Tensor and records its parents. Today, just the forward part of + and *: compute the values and remember the parents.
Tensor: in deep learning, simply an n-dimensional array. Here, an array that also carries a gradient and its history.
Words and symbols in this lesson
- Tensor
- here: an array plus its gradient plus the Tensors it came from
- .grad
- the gradient of the final loss with respect to this Tensor
- unbroadcast
- summing a gradient back down to the shape it was stretched from
Fill in __add__ and __mul__ so they return a new Tensor whose data is the sum or product, and whose parents is (self, other). Plain numbers should work too, via Tensor.lift. Leave _backward alone for now.
Spotted a problem in this lesson? Report it · Ask about this lesson
This exercise opens soon
Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.
This browser is blocking cookies for learn.floati.life, so lessons you finished here cannot open in order. Allow cookies for this site and reload.