Archives for SGD

01 Apr

What is momentum in a Neural network and how does it work?

In a neural network, there is the concept of loss, which is used to calculate performance. The higher the loss, the poorer the performance of the neural network, that is why we always try to minimize the loss so that the neural network performs better.

13 Mar

A beginner’s guide to stochastic gradient descent from scratch

Yugesh Verma artificial intelligence

Stochastic gradient descent is also a method of optimization. The objective function which needs to be optimised comes with suitable smoothness properties and this suitable smoothness makes the stochastic gradient descent different from the gradient descent.

03 Jun

Does Deep Learning Suffer From Too Many Optimizers?

Ram Sagar ADAM

“There is no single optimizer that dominates its competitors across all tasks.” Critics often call machine learning ‘glorified statistics’. There is some merit to the argument. The fundamental function of any machine learning model is pattern recognition, which relies on the principles of convergence; the methods of fitting data to the model. To that end,…

The post Does Deep Learning Suffer From Too Many Optimizers? appeared first on Analytics India Magazine.

12 Sep

A Lowdown On Alternatives To Gradient Descent Optimization Algorithms

Ram Sagar Adagrad

Gradient Descent is the most common optimisation strategy used in machine learning frameworks. It is an iterative algorithm used to minimise a function to its local or global minima. In simple words, Gradient Descent iterates overs a function, adjusting it’s parameters until it finds the minimum. Gradient Descent is used to minimise the error by…

The post A Lowdown On Alternatives To Gradient Descent Optimization Algorithms appeared first on Analytics India Magazine.

21 Sep

How Stochastic Gradient Descent Is Solving Optimisation Problems In Deep Learning

Richa Bhatia deep learning

To a large extent, deep learning is all about solving optimisation problems. According to computer science researchers, stochastic gradient descent, better known as SGD has become the workhorse of Deep Learning, which, in turn, is responsible for the remarkable progress in computer vision. Despite its simplicity, SGD is a simple variant of classical gradient descent […]

The post How Stochastic Gradient Descent Is Solving Optimisation Problems In Deep Learning appeared first on Analytics India Magazine.