∫ntegrabℓε ∂iﬀerentiαℓs · unorganised notes, code, and writings of random topics

Bengio et al (2003) A Neural Probabilistic Language Model

April 8, 2022 paper

Language model has the curse of dimensionality, namely, if we want to model the joint distribution of 10 words in a vocabulary of \(10^5\), there are \(10^{5\times 10} = 10^{50}\) parameters. But if the variables are continuous, we can use fewer parameters. [more]

Sampling and Visualizing Copulae

April 2, 2022 blog code math

Copula is a multivariate CDF defined over \(C: [0,1]^d \mapsto [0,1]\), [more]

Gaussian Process Regression

March 15, 2022 blog math

Gaussian process regression is an interesting method of function regression. It is non-parametric, meaning not to assume anything (linear, polynomial, etc.) about the objective function. Let’s consider a function of \(\mathbb{R}\mapsto\mathbb{R}\). Without knowing anything or assuming anything, the fuction \(f(x)\) can take, and maps to, any value in \(\mathbb{R}\). We... [more]

Upgrading OpenWRT

March 4, 2022 blog system

At the time when I bought my Linksys WiFi routers, OpenWRT was in the version 19.x and now we have version 21.x. I replaced DD-WRT to OpenWRT in the past and I would not expect the DD-WRT config can be port to OpenWRT. But for upgrading the versions of OpenWRT,... [more]

Different models of ARIMA

February 25, 2022 blog math

In statsmodels, there are several different but similar functions to fit an ARIMA model to the data. We have the ARIMA, SARIMAX and AutoReg classes. While they are all usable, and they are all fit using OLS, the model they produced are slightly different and hence the fitted parameters are... [more]