Gate: make an untrainable deep net train (AI off)
Why it matters Fixing a deep network that will not learn is one of the most common real jobs in this field. Here you do it alone.
AI tutor off. The 30-layer network below never leaves a coin toss. You may change how it is started and how its blocks are wired, using anything from this module, but keep 30 hidden layers of width 16 and 300 steps of plain SGD.
Words and symbols in this lesson
- Xavier, LeCun, He
- starting spreads sqrt(2 / (inputs + outputs)) or 1 / sqrt(inputs) for tanh, and sqrt(2 / inputs) for ReLU
Edit build(rng) and forward(params, X) so that training reaches at least 90% training accuracy on the moons.
AI off: no tutor, hints or worked solution on this one. Your own thinking only.
Spotted a problem in this lesson? Report it · Ask about this lesson
This exercise opens soon
Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.
This browser is blocking cookies for learn.floati.life, so lessons you finished here cannot open in order. Allow cookies for this site and reload.