$$ \require{physics} \newcommand{B}{\mathcal{B}}$$

Batch Normalization was proposed by Ioffe and Szegedy in 2015, and it spawned several normalization techniques that are used in SOTA models today (layer norm, weight norm, etc). Batch normalization normalizes the output of each layer based on the mean and variance of the …

Continue reading

A nice intuition for L2 regularization comes from having a prior on the distribution of parameters: the prior assumes that the parameters are close to zero. Let's assume that the prior is $\mathcal{N}(0, \Sigma)$. The MAP estimate of the parameters would then be

$$\begin{align} \theta_{\text{MAP …

Continue reading

$\newcommand{\triple}{(\Omega, \mathcal{F}, \mathbf{P})}$$\newcommand{\P}{\mathbf{P}}$

Recap

If you remember the note on random variables, then this picks up exactly where that left off.

Definition 1.1: A Measurable space $(X,\Sigma)$ consists of a set $X$ and a $\sigma$-algebra $\Sigma$ defined on $X …

Continue reading

I promised I wouldn't look at the JEE 2022 paper, but couldn't help sneaking a peek. As always, the professors currently making life difficult occasionally get a chance to make the life of all of India's students difficult, and they never cease to amaze.

Welp, this will be a short …

Continue reading

$\newcommand{\cE}[2]{\mathbf{E}(#1\ |\ #2)}$$\newcommand{\cP}[2]{\mathbf{P}(#1\ |\ #2)}$$\renewcommand{\P}[1]{\mathbf{P}(#1)}$$\newcommand{\E}[1]{\mathbf{E}(#1)}$$\newcommand{\F}{\mathcal{F}}$$\newcommand{\G}{\mathcal{G}}$$\newcommand{\ind}[1]{\mathbf{1}_{#1}}$ To motivate this note, I’ll pose the following …

Continue reading

$\newcommand{\triple}{(\Omega, \mathcal{F}, \mathbf{P})}$$\newcommand{\P}{\mathbf{P}}$ This note on random variables follows as a result of confusing notation in several math textbooks. I'll explain random variables (in measure theoretic terms) as verbosely as I can, and then prove some results. This article assumes that the …

Continue reading

I'll begin this article by brushing up a few definitions:

A function is a mapping between two sets: the domain D and the codomain C.

It's very important to note here that the function is the mapping itself, and not an element in the codomain or the domain. The function …

Continue reading

This is a short getting started article on LaTeX; Recently, one of our courses involved a bit of LaTeX work, and this is meant to be a short introduction on how to use LaTeX to explain one's working.

Getting Started

LaTeX (Pronounced lay-tech, stylized $\LaTeX$), at it's core, is merely …

Continue reading

$\newcommand{\pdv}[2]{\frac{\partial{#1}}{\partial{#2}}}$ $\newcommand{\ah}{\pmb{a} + \pmb{h}}$ $\newcommand{\a}{\pmb{a}}$ $\newcommand{\h}{\pmb{h}}$

The Jacobian Matrix

Consider a function that maps reals to reals, $f:\Bbb{R} \to \Bbb{R}$. The linear approximation of this function is given by $$f(a …

Continue reading

Just after 10th and as I was beginning my JEE Preparation, I felt the need for an extensible note-taking apparatus. Online notes would be too much trouble and would keep me hooked to the computer. Notebooks were also difficult, as I wanted my notes to be extensible; adding pages in …

Continue reading

This beauty came in the 2nd September shift 2 paper:

Let $A = \{ X = (x, y, z)^T : PX = 0 \text{ and } x^2+y^2+z^2=1 \}$, where $$P = \left[ \begin{array}{l l l}1&2&1 \\ -2&3&-4 \\ 1&9&-1 \end{array}\right]$$ then the …

Continue reading

This is a small set of posts that come into the category of "notes": self-explanations of concepts that I have recently picked up and found interesting.

Lagrange Interpolation

Lagrange Interpolation is a concept that allows us to find a polynomial of least degree passing through a given set of points …

Continue reading

My old man always used to say that there are two ways to solve a problem: there's the horse way, and there's the donkey way. The donkey way involves tedious calculations, whereas the horse way 'cuts' through the problem. Take this problem as an example:

For each positive integer $n …

Continue reading

The ISC Exam results were released on 10th June, 3 PM IST. In this article, I'll be analyzing the results of the 117 students in the science stream of my school and showing how they performed. I'll also weave a story with the data and point out things that could …

Continue reading

A long time ago, I gave my friend (Let's call him C) an integral to solve. He came back to me a few days later and the conversation went something like this:

C: Debu, I couldn't solve that African integral you gave me.
Me: Ok, but why are you calling …

Continue reading

This list consists of the limits that I found most challenging.

1. $$\lim_{n \to \infty} \left( \frac{n!}{n^n} \right) ^\frac 1n$$
2. $$\lim_{x \to 0} \left( \frac{1}{\ln(x + \sqrt{x^2+1})} - \frac 1{\ln(x+1)} \right)$$
3. $$\lim_{n \to \infty} \frac{n + n^2 …

Continue reading

Simple limit problems consist of the form $\lim_{x \to 0}\frac{O_1(x)O_2(x)..}{O_a(x)O_b(x)..}$, such as $\lim_{x \to 0} \frac{\sin 3x \tan 2x \tan^{-1} 5x}{x^2 \ln(1+x)}$. Here, the infinitesimals are well defined and cancel out easily. Some …

Continue reading

The USA currently stands at 1.76 million COVID-19 cases. That's more than the next 5 nations combined. A large number of these cases are due to government inaction against the virus. The lack of a concerted lockdown across the country is also to blame. Here's a graph showing the …

Continue reading

The Previous COVID regression analysis was fairly accurate. However, the opening of lockdown offset the statistics a bit and now there are more number of projected cases. Here is a recomputation of the statistics, which projects an average of 172,000 cases by June 1 and 520,000 cases overall …

Continue reading

This is an interesting number theory fact that seems strange when taken at face value. Here's a small proof of it:

Continue reading