python (12.9k questions)

javascript (9.2k questions)

reactjs (4.7k questions)

java (4.2k questions)

java (4.2k questions)

c# (3.5k questions)

c# (3.5k questions)

html (3.3k questions)

Questions - gradient-descent

Linear Regression Stochastic Gradient Descent

I am trying to fit a sinusoidal wave (sin(2 pi x)) with some gaussian noise added to it. I am using the stochastic gradient descent algorithm, and the model I am trying to fit is linear in the paramet...
test-img

Karthik Prakash

python

numpy

machine-learning

linear-regression

gradient-descent

Votes: 0

Answers: 1

Latest Answer

This can be an issue of a too large step size/learning rate a. The gradient you are computing is just a noisy version of the true gradient. If your step size is too large, your just jumping around alm...
test-img

Simon Hawe

torch.no_grad() and detach() combined

I encountered many code fragments like the following for choosing an action, that include a mix of torch.no_grad and detach (where actor is some actor, SomeDistribution your preferred distribution), a...
test-img

joinijo

machine-learning

deep-learning

neural-network

pytorch

gradient-descent

Votes: 0

Answers: 1

Latest Answer

While it may be redundant, it depends on the internals of actor and SomeDistribution. In general, there are three cases I can think of where detach would be necessary in this code. Since you've alread...
test-img

jodag

Gradient only defined for scalar-output functions. Output had shape: (1,)

I'm working with gradients and having some troubles, here is my code def model(x): return (x+1)**2 + (x-1)**2 def loss(x,y): return y-model(x) x=2 grad = jax.grad(loss,argnums=0) gradient = grad...
test-img

Lorenzo Cutrupi

python

gradient

gradient-descent

Votes: 0

Answers: 1

Latest Answer

The grad function requires real or complex-valued inputs but you are using integers. This chunk will work import jax def model(x): return (x+1)**2 + (x-1)**2 def loss(x, y): return y - model(x) ...
test-img

amarchin

Understanding Pytorch Weight and Biases in Linear Layer

Below is the code for combining weight and bias into a single layer, I am not able to understand the line below, why we have to multiply weight transpose matrix with bais. I should just bias without w...
test-img

ZKS

python

deep-learning

neural-network

pytorch

gradient-descent

Votes: 0

Answers: 1

Latest Answer

You simply need to expand the original equation of two Linear layers i.e. # out = layer2(layer1(x)) # given (x @ A + B) @ C + D out = (x @ layer1.weight.t() + layer1.bias) @ layer2.weight.t() + layer2...
test-img

kHarshit

Posts

Questions

Blogs

Jobs