Batch Normalization

Posted on Thu 16 February 2023 in Mathematics • Tagged with Mathematics, Machine Learning, Deep Learning

$$ \require{physics} \newcommand{B}{\mathcal{B}}$$

Batch Normalization was proposed by Ioffe and Szegedy in 2015, and it spawned several normalization techniques that are used in SOTA models today (layer norm, weight norm, etc). Batch normalization normalizes the output of each layer based on the mean and variance of the …


Continue reading

L2 regularization intuition

Posted on Sun 22 January 2023 in Mathematics • Tagged with Mathematics, Machine Learning, Deep Learning

A nice intuition for L2 regularization comes from having a prior on the distribution of parameters: the prior assumes that the parameters are close to zero. Let's assume that the prior is $\mathcal{N}(0, \Sigma)$. The MAP estimate of the parameters would then be

$$\begin{align} \theta_{\text{MAP …


Continue reading

Filtrations

Posted on Mon 28 November 2022 in Mathematics • Tagged with Mathematics

$\newcommand{\triple}{(\Omega, \mathcal{F}, \mathbf{P})}$$\newcommand{\P}{\mathbf{P}}$

Recap

If you remember the note on random variables, then this picks up exactly where that left off.

Definition 1.1: A Measurable space $(X,\Sigma)$ consists of a set $X$ and a $\sigma$-algebra $\Sigma$ defined on $X …


Continue reading

The JEE never fails to amaze

Posted on Mon 19 September 2022 in Mathematics • Tagged with Mathematics, JEE

I promised I wouldn't look at the JEE 2022 paper, but couldn't help sneaking a peek. As always, the professors currently making life difficult occasionally get a chance to make the life of all of India's students difficult, and they never cease to amaze.

Welp, this will be a short …


Continue reading

A Note on Conditional Probability

Posted on Sat 25 December 2021 in Mathematics • Tagged with Mathematics

$\newcommand{\cE}[2]{\mathbf{E}(#1\ |\ #2)}$$\newcommand{\cP}[2]{\mathbf{P}(#1\ |\ #2)}$$\renewcommand{\P}[1]{\mathbf{P}(#1)}$$\newcommand{\E}[1]{\mathbf{E}(#1)}$$\newcommand{\F}{\mathcal{F}}$$\newcommand{\G}{\mathcal{G}}$$\newcommand{\ind}[1]{\mathbf{1}_{#1}}$ To motivate this note, I’ll pose the following …


Continue reading

A Note on Random Variables

Posted on Fri 03 December 2021 in Mathematics • Tagged with Mathematics

$\newcommand{\triple}{(\Omega, \mathcal{F}, \mathbf{P})}$$\newcommand{\P}{\mathbf{P}}$ This note on random variables follows as a result of confusing notation in several math textbooks. I'll explain random variables (in measure theoretic terms) as verbosely as I can, and then prove some results. This article assumes that the …


Continue reading

Function transforms (providing a broader picture of Laplace Transforms)

Posted on Wed 12 May 2021 in Mathematics • Tagged with Mathematics

I'll begin this article by brushing up a few definitions:

A function is a mapping between two sets: the domain D and the codomain C.

function definition

It's very important to note here that the function is the mapping itself, and not an element in the codomain or the domain. The function …


Continue reading

Understanding Jacobians

Posted on Fri 29 January 2021 in Mathematics • Tagged with Mathematics

$\newcommand{\pdv}[2]{\frac{\partial{#1}}{\partial{#2}}}$ $\newcommand{\ah}{\pmb{a} + \pmb{h}}$ $\newcommand{\a}{\pmb{a}}$ $\newcommand{\h}{\pmb{h}}$

The Jacobian Matrix

Consider a function that maps reals to reals, $f:\Bbb{R} \to \Bbb{R}$. The linear approximation of this function is given by $$f(a …


Continue reading

Setting up MathNotes - an online math repository

Posted on Fri 16 October 2020 in Mathematics • Tagged with Mathematics, Programming

Just after 10th and as I was beginning my JEE Preparation, I felt the need for an extensible note-taking apparatus. Online notes would be too much trouble and would keep me hooked to the computer. Notebooks were also difficult, as I wanted my notes to be extensible; adding pages in …


Continue reading

A Good JEE Main (September) problem

Posted on Fri 11 September 2020 in Mathematics • Tagged with Mathematics, JEE

This beauty came in the 2nd September shift 2 paper:

Let $A = \{ X = (x, y, z)^T : PX = 0 \text{ and } x^2+y^2+z^2=1 \}$, where $$P = \left[ \begin{array}{l l l}1&2&1 \\ -2&3&-4 \\ 1&9&-1 \end{array}\right]$$ then the …


Continue reading

Math Notes: Lagrange interpolation

Posted on Fri 14 August 2020 in Mathematics • Tagged with Mathematics, Math Notes

This is a small set of posts that come into the category of "notes": self-explanations of concepts that I have recently picked up and found interesting.

Lagrange Interpolation

Lagrange Interpolation is a concept that allows us to find a polynomial of least degree passing through a given set of points …


Continue reading

Is the shortest solution to a problem the best?

Posted on Sun 26 July 2020 in Mathematics • Tagged with Mathematics

My old man always used to say that there are two ways to solve a problem: there's the horse way, and there's the donkey way. The donkey way involves tedious calculations, whereas the horse way 'cuts' through the problem. Take this problem as an example:

For each positive integer $n …


Continue reading

ISC 2020 Analysis: An application of Data Science and Statistics

Posted on Sat 11 July 2020 in Mathematics • Tagged with Mathematics, ISC, Data Science

The ISC Exam results were released on 10th June, 3 PM IST. In this article, I'll be analyzing the results of the 117 students in the science stream of my school and showing how they performed. I'll also weave a story with the data and point out things that could …


Continue reading

Solving the African Integral (from YG file cover)

Posted on Thu 09 July 2020 in Mathematics • Tagged with Mathematics

A long time ago, I gave my friend (Let's call him C) an integral to solve. He came back to me a few days later and the conversation went something like this:

C: Debu, I couldn't solve that African integral you gave me.
Me: Ok, but why are you calling …


Continue reading

A Compilation of hard limits

Posted on Fri 05 June 2020 in Mathematics • Tagged with Mathematics

This list consists of the limits that I found most challenging.

  1. $$\lim_{n \to \infty} \left( \frac{n!}{n^n} \right) ^\frac 1n$$
  2. $$\lim_{x \to 0} \left( \frac{1}{\ln(x + \sqrt{x^2+1})} - \frac 1{\ln(x+1)} \right)$$
  3. $$\lim_{n \to \infty} \frac{n + n^2 …

Continue reading

Limit involving higher order infinitesimals

Posted on Fri 05 June 2020 in Mathematics • Tagged with Mathematics

Simple limit problems consist of the form $\lim_{x \to 0}\frac{O_1(x)O_2(x)..}{O_a(x)O_b(x)..}$, such as $\lim_{x \to 0} \frac{\sin 3x \tan 2x \tan^{-1} 5x}{x^2 \ln(1+x)}$. Here, the infinitesimals are well defined and cancel out easily. Some …


Continue reading

COVID-19 USA Analysis: effects of the lack of lockdown

Posted on Fri 29 May 2020 in Mathematics • Tagged with Mathematics, COVID 19

The USA currently stands at 1.76 million COVID-19 cases. That's more than the next 5 nations combined. A large number of these cases are due to government inaction against the virus. The lack of a concerted lockdown across the country is also to blame. Here's a graph showing the …


Continue reading

COVID 19 Regression analysis Update

Posted on Tue 19 May 2020 in Mathematics • Tagged with Mathematics, COVID 19

The Previous COVID regression analysis was fairly accurate. However, the opening of lockdown offset the statistics a bit and now there are more number of projected cases. Here is a recomputation of the statistics, which projects an average of 172,000 cases by June 1 and 520,000 cases overall …


Continue reading

No square ends in 3

Posted on Fri 15 May 2020 in Mathematics • Tagged with Mathematics

This is an interesting number theory fact that seems strange when taken at face value. Here's a small proof of it:


Continue reading

COVID 19 regression analysis

Posted on Wed 06 May 2020 in Mathematics • Tagged with Mathematics, COVID 19

regression This is a regression analysis attempt for the COVID-19 spread data. The graphs represent total cases per day. The Orange graph is USA, the smaller graph on the left is China and the graph on the right is India. A Standard gaussian curve of the form $Ae^{-b(x-c)^2 …


Continue reading