How Deep Learning libraries handle gradients — How do you compute derivatives of a function? By hand, you’d find it analytically. You can then write a function to compute it. Or you can write a function to approximate it. def f(x):
return x*x // function def df(x):
return 2*x // analytical derivative def df(x,f,e):
return (f(x+e) - f(x))/e // approximation This can get quite cumbersome…