Conda and CUDA

July 12, 2023 blog system

If we want to run TensorFlow or PyTorch with CUDA on Linux, for example, we can install CUDA as a system library first and then install the Python package with pip (or via apt-get, in the rare case). This way, the package will find the CUDA library at system locations.... [more]

Wang et al (2021) Real-ESRGAN

June 10, 2023 paper

This is to extend the SR-GAN and ESR-GAN to do blind super-resolution. The problem statement is to reconstruct the high-resolution image from low resolution (a.k.a. super-resolution), but without knowing how the low resolution is derived from the original high-resolution image, i.e., blind super-resolution. [more]

Explaining Attention Mechanism

May 21, 2023 blog math

Attention mechanism was first mentioned in Bahdanau et al (2015) paper titled “Neural Machine Translation by Jointly Learning to Align and Translate”, and Luong et al (2015) improved it with the paper “Effective Approaches to Attention-based Neural Machine Translation”. The key is to find the attention score $a_{ij}$ between two... [more]