Evolutionary Methods in Compression
Evolutionary algorithms are heruistic-based optimization algorithms inspired by biological evolution (Darwin, 1859). In a broad scope, it involves crafting y...
Blog
Recent Posts
LayerDrop Regularisation + Knowledge Distillation?
Regularisation techniques such as Dropout is a powerful in preventing over-fitting of models during training. However, another benefit it confers is this- ap...
Compression as Feature Selection
Feature selection is a popular machine learning technique used to identify subsets of relevant features (e.g. variables, predictors) in model construction, w...
Lottery Ticket Hypothesis
The Lottery Ticket Hypothesis is a really intersting piece of research, clinching the Best Paper Award in ICLR 2019. In this post, we delve into the LTH and ...
Binary Neural Networks
Binary Neural Networks (BNNs) are an extreme form of quantization in neural networks, where the weights are represented as binary digits taking on the values...
Compression Methods
There are plethora of model compression techniques used to compress very large trained neural networks for inference on edge devices, such as mobile phones a...
Latent Tree Induction #2
This post is a continuation of my latent tree induction research notes.
Latent Tree Induction #1
Introduction Now that it has been over a year of pursuing a PhD in AI, let’s do a quick stock-take of what I have learnt thus far. Having come into the progr...
Additive Attention Mechanism
Attention mechanism is a very popular technique used in neural models today, with many powerful variations. Today, we will look at additive attention (Bahdan...
Sequence to Sequence Modeling
Today, we look a historic model: the encoder-decoder architecture in sequence-to-sequence modeling (Sutskever et al., 2014). In this post, we will walk throu...
Project Euler #6-10
Continuing on from the Project Euler Challenge, we look at the next 5 problems in the set.
Project Euler #1-5
To keep my algorithmic programming skills in shape, I decided to pick up the Project Euler Challenge. This daily training in computational thinking and debug...
Introduction
This blog was created to build a habit of critically reading AI research papers and brainstorming ideas. I’m hoping that the action of posting my reflections...