Improving Deep Neural Networks
Hyperparameter Tuning, Regularization and Optimization

please refer to this video -> https://www.coursera.org/learn/deep-neural-network/lecture/y0m1f/gradient-descent-with-momentum for the concept of momentum in machine learning. The momentum simply uses the concept of running averages for making the updates. This equation is different from Momentum in two respects: 1.) this doesn't aggregate the gradients; and 2.) This has the cost term in the update rule which is not there in momentum or any other gradient descent variant as a matter of fact.

Thank you for the feedback.

https://www.coursera.org/learn/deep-neural-network/lecture/XjuhD/bias-correction-in-exponentially-weighted-averages

If you've never had a course I suggest you try some that focus on concepts like overfitting /underfitting and their relationship with the data you feed your model and the hyperparams you choose.

As a reference, take a look at this book:

https://www.deeplearning.ai/machine-learning-yearning/

And these courses:

https://www.coursera.org/learn/deep-neural-network?specialization=deep-learning

https://www.coursera.org/learn/machine-learning-projects?specialization=deep-learning