∫ntegrabℓε ∂iﬀerentiαℓs · unorganised notes, code, and writings of random topics

Explaining Attention Mechanism

May 21, 2023 blog math

Attention mechanism was first mentioned in Bahdanau et al (2015) paper titled “Neural Machine Translation by Jointly Learning to Align and Translate”, and Luong et al (2015) improved it with the paper “Effective Approaches to Attention-based Neural Machine Translation”. The key is to find the attention score $a_{ij}$ between two... [more]

Carion et al (2020) End-to-End Object Detection with Transformers

May 20, 2023 paper

Object detection is to predict the bounding boxes and category labels for each object of interest. This paper proposed DETR (Detection Transformer) to predict all objects at once, trained end-to-end with a set loss function to perform bipartite matching between the predicted and groundtruth. It is found to perform better... [more]

Prokhorenkova et al (2018) CatBoost: Unbiased boosting with categorical features

May 19, 2023 paper

CatBoost is a library for random forest. This paper describes the key feature behind it. [more]

Redmon et al (2016) You Only Look Once: Unified, Real-time Object Detection

April 2, 2023 paper

This is the paper to propose YOLOv1 network, which reframed object detection as a regression problem. It is a single convolutional network that simultaneously predicts multiple bounding boxes and class probabilities for those boxes. It is to compare against R-CNN but faster and can see the entire image at once.... [more]

Liu et al (2016) SSD: Single Shot MultiBox Detector

April 1, 2023 paper

This paper distinct from previous work in the sense that the older approach of object detection first hypothesize bounding boxes, resample features for each box, then apply a classifier. This paper proposed a network that does not resample for bounding box hypotheses but equally accurate. It can do high speed... [more]