高斯分布的微分熵

$X \sim \mathcal{N}(\mu, \sigma^2)~$，$\displaystyle f(x)=\frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{(x-\mu)^2}{2\sigma^2}}$，其微分熵推导过程如下：

$$\begin{split} h(X) &= \int_{-\infty}^{\infty} \frac{-1}{\sqrt{2\pi\sigma^2}}\exp\left(-\frac{(x-\mu)^2}{2\sigma^2}\right) \cdot \ln \left[ \frac{1}{\sqrt{2\pi\sigma^2}}\exp\left(-\frac{(x-\mu)^2}{2\sigma^2}\right) \right] dx \\[4pt] &= \frac{-1}{\sqrt{2\pi\sigma^2}} \int_{-\infty}^{\infty} \exp\left(-\frac{(x-\mu)^2}{2\sigma^2}\right) \cdot \left( \ln(2\pi\sigma^2)^{-1/2} - \frac{(x-\mu)^2}{2\sigma^2} \right) dx \\[4pt] &= \frac{1}{2}\ln(2\pi\sigma^2) \int_{-\infty}^{\infty} \frac{1}{\sqrt{2\pi\sigma^2}}\exp\left(-\frac{(x-\mu)^2}{2\sigma^2}\right) dx \\[4pt] &\quad + \int_{-\infty}^{\infty} \frac{(x-\mu)^2}{2\sigma^2} \cdot \frac{1}{\sqrt{2\pi\sigma^2}}\exp\left(-\frac{(x-\mu)^2}{2\sigma^2}\right) dx \\[4pt] &= \frac{1}{2}\ln(2\pi\sigma^2) + \frac{1}{2\sigma^2}E(X-\mu)^2 \\[4pt] &= \frac{1}{2}\ln(2\pi\sigma^2) + \frac{1}{2} = \frac{1}{2}\ln(2\pi e\sigma^2) \qquad\text{(nats)} \end{split}$$

又因为 $H_b(X) = \log_ba \cdot H_a(X)$，于是取 $b=e, ~a=2$

$$\implies \frac{1 \text{ nat}}{x \text{ bits}} = \ln2 \implies x = \frac{1}{\ln2} = \frac{\log e}{\log2} = \log e \text{ bits}$$

所以

$$\frac{1}{2}\ln(2\pi e\sigma^2) \text{ nats} = \frac{1}{2} \cdot \frac{\log(2\pi e\sigma^2)}{\log e} \cdot \log e \text{ bits} = \frac{1}{2}\log(2\pi e\sigma^2) \text{ bits}$$