L2 regularization intuition

Posted on Sun 22 January 2023 in Mathematics • Tagged with Mathematics, Machine Learning, Deep Learning

A nice intuition for L2 regularization comes from having a prior on the distribution of parameters: the prior assumes that the parameters are close to zero. Let's assume that the prior is $\mathcal{N}(0, \Sigma)$. The MAP estimate of the parameters would then be

$$\begin{align} \theta_{\text{MAP …


Continue reading

Optimizers, Part 1

Posted on Mon 02 January 2023 in Programming • Tagged with Programming, Machine Learning, Deep Learning

Happy New Year! This is going to was supposed to be a long one, so sit back and grab a chocolate (and preferably view this on your laptop)

Some optimization algorithms. Click on a colour in the legend to hide/show it

Table of Contents

  1. Introduction

Continue reading

My DL workflow for 2023

Posted on Thu 29 December 2022 in Programming • Tagged with Programming, Machine Learning, Deep Learning

I've kind of zeroed down on Deep Learning at this point, and putting my money where my mouth is, will be taking both COL772 (Natural Language Processing) and COL775 (Deep Learning) next semester.

Along with Operating Systems, Parallel Programming and Theory of Computation.

Why a workflow?

I'll need to train …


Continue reading

Variance Estimation in Heteroscedastic Regression

Posted on Mon 29 August 2022 in Programming • Tagged with Programming, Machine Learning

Most of the times, the regression models we need to fit don't have a constant standard deviation, but rather one that is some function of $x$. In this article, I try to learn a model with variance varying linearly as a function of x: kind of like a regression on variance


Continue reading