Explaining Attention Mechanism

Attention mechanism was first mentioned in Bahdanau et al (2015) paper titled “Neural Machine Translation by Jointly Learning to Align and Translate”, and Luong et al (2015) improved it with the paper “Effective Approaches to Attention-based Neural Machine Translation”. The key is to find the attention score $a_{ij}$ between two... [more]

Carion et al (2020) End-to-End Object Detection with Transformers

Object detection is to predict the bounding boxes and category labels for each object of interest. This paper proposed DETR (Detection Transformer) to predict all objects at once, trained end-to-end with a set loss function to perform bipartite matching between the predicted and groundtruth. It is found to perform better... [more]

Liu et al (2016) SSD: Single Shot MultiBox Detector

This paper distinct from previous work in the sense that the older approach of object detection first hypothesize bounding boxes, resample features for each box, then apply a classifier. This paper proposed a network that does not resample for bounding box hypotheses but equally accurate. It can do high speed... [more]