Summer-Analytics-2022

This repository contains the course notes, implementations of various ML models and Hackathon submission notebooks of Summer Analytics program.

---

Project maintained by HridayAg0102 Hosted on GitHub Pages — Theme by mattgraham

Regularization

Reference Links:

• Videos 7.1 to 7.4

---

Ways to Address Overfitting:

1. Reduce Number of Features.
   • Manually select which features to keep.
   • Model Selection algorithms.
2. Regularization
   • Keep all the features, but reduce magnitude/values of parameters $\large \theta_j$
   • Works well if we have lots of features, each of which contributes a bit in predicting y (output)

Regularization

In regularization, we try to penalize the parameters within the cost function with some high values so that when we
miniize the value of J($\theta$), we get the effect of those parameters reduced.

In this formula, $\Huge \lambda$ refers to Regularization Parameter.

What Happens when we choose Extremely High value of Lambda ?

Thus, for very large value of $\Large \lambda$, we almost neglect the effect of panalized parameters which may
lead to Underfitting Model predictions.

---

👉 We Don’t panalize $\Huge \theta_0$ by convention. Though, even if it is panalized, it puts negligibly small effect on the trained model

---

▶️ Regularization in Linear Regression

1. In Gradient Descent

The Term $\Huge \left( 1-\alpha\frac{\lambda}{m} \right)$ < 1

2. In Normal Equation Method

:star: Problem of Non Invertibility in Normal Equation Method (Doesn’t occur while using Regularization)

---

▶️ Regularization in Logistic Regression

Examples of Regularization:

• K-means: Restricting the segments for avoiding redundant groups.
• Neural networks: Confining the complexity (weights) of a model.
• Random Forest: Reducing the depth of tree and branches (new features)

---

⏭️ L1 and L2 Regularization {Part 2 of these notes. are present at this link.}