Neyman Factorization Theorem

1. Statement

Theorem (Neyman Factorization Theorem). A statistic $S(\mathbf{X})$ is a sufficient statistic for $\theta$ if and only if the probability density function can be written as

$$f(\mathbf{x},\theta )=h(\mathbf{x})g(S(\mathbf{x}),\theta )$$

for some functions $h(\mathbf{x})$ and $g(S(\mathbf{x}),\theta )$.

2. Examples

Example (Sufficient statistic for Bernoulli distribution). Consider $X_1,\cdots ,X_n{\sim }_{i.i.d.}\text{Bernoulli}(p)$. The likelihood function is

$$L(p)=\prod _{i=1}^n{p}^{x_i}(1-p{)}^{1-x_i}=p^{{\sum }_{i=1}^n{x}_i}(1-p{)}^{n-{\sum }_{i=1}^n{x}_i}.$$

Let $S(\mathbf{X})={\sum }_{i=1}^n{X}_i$. Then the likelihood function can be written as

$$L(p)=p^{S(\mathbf{X})}(1-p{)}^{n-S(\mathbf{X})}=\left(p^{S(\mathbf{X})}\right)\left((1-p{)}^{n-S(\mathbf{X})}\right).$$

This is a product of two functions: one is a function of $S(\mathbf{X})$ only, and the other is a function of $\mathbf{X}$ only. Therefore, by the Neyman Factorization Theorem, $S(\mathbf{X})$ is a sufficient statistic for $p$.

Example (Sufficient statistic for Poisson distribution). Consider $X_1,\cdots ,X_n{\sim }_{i.i.d.}\text{Poisson}(\lambda )$. The likelihood function is

$$L(\lambda )=\prod _{i=1}^n\frac{{\lambda }^{x_i}{e}^{-\lambda }}{x_i!}=\frac{{\lambda }^{{\sum }_{i=1}^n{x}_i}{e}^{-n\lambda }}{{\prod }_{i=1}^n{x}_i!}.$$

Let $S(\mathbf{X})={\sum }_{i=1}^n{X}_i$. Then the likelihood function can be written as

$$L(\lambda )={\lambda }^{S(\mathbf{X})}{e}^{-n\lambda }\cdot \frac{1}{{\prod }_{i=1}^n{x}_i!}:=g(S(\mathbf{X}),\lambda )\cdot h(\mathbf{X}).$$

Therefore, by the Neyman Factorization Theorem, $S(\mathbf{X})$ is a sufficient statistic for $\lambda$.

Example (Sufficient statistic for normal distribution with ${\sigma }^2$ known). Consider $X_1,\cdots ,X_n{\sim }_{i.i.d.}\text{Normal}(\mu ,{\sigma }^2)$ with ${\sigma }^2$ known. The likelihood function is

$$L(\mu )={\left(\frac{1}{\sqrt{2\pi {\sigma }^2}}\right)}^n\exp \left\{-\frac{1}{2{\sigma }^2}\sum _{i=1}^n(x_i-\mu {)}^2\right\}.$$

Let $S(\mathbf{X})={\sum }_{i=1}^n{X}_i$. Then the likelihood function can be written as

$$\begin{array}{rl}L(\mu )& =\underset{g(\sum \text{X},\mu )(\text{with }\sigma ,n\text{ known})}{\underbrace{{\left(\sqrt{2\pi {\sigma }^2}\right)}^{-n}\cdot \exp \left\{-\frac{n{\mu }^2}{2{\sigma }^2}\right\}\cdot \exp \left\{\frac{\mu {\sum }_{i=1}^n{x}_i}{{\sigma }^2}\right\}}}\\ & \cdot \underset{h(\mathbf{X})(\text{function of }\mathbf{X})}{\underbrace{\exp \left\{-\frac{{\sum }_{i=1}^n{x}_i^2}{2{\sigma }^2}\right\}}}\\ & =g(S(\mathbf{X}),\mu )\cdot h(\mathbf{X}).\end{array}$$

Therefore, by the Neyman Factorization Theorem, $S(\mathbf{X})={\sum }_{i=1}^n{X}_i$ is a sufficient statistic for $\mu$.

Example (Sufficient statistic for normal distribution with ${\sigma }^2$ unknown). Consider $X_1,\cdots ,X_n{\sim }_{i.i.d.}\text{Normal}(\mu ,{\sigma }^2)$ with both $\mu$ and ${\sigma }^2$ unknown. The likelihood function is

$$L(\mu ,{\sigma }^2)={\left(\frac{1}{\sqrt{2\pi {\sigma }^2}}\right)}^n\exp \left\{-\frac{1}{2{\sigma }^2}\sum _{i=1}^n(x_i-\mu {)}^2\right\}.$$

Let $S(\mathbf{X})=\left({\sum }_{i=1}^n{x}_i,{\sum }_{i=1}^n{x}_i^2\right)$. Then the likelihood function can be written as

$$\begin{array}{rl}L(\mu )& =\underset{g(\sum x_i,\sum x_i^2,\mu )}{\underbrace{{\left(\sqrt{2\pi {\sigma }^2}\right)}^{-n}\cdot \exp \left\{-\frac{n{\mu }^2}{2{\sigma }^2}\right\}\cdot \exp \left\{\frac{\mu {\sum }_{i=1}^n{x}_i}{{\sigma }^2}\right\}\cdot \exp \left\{-\frac{{\sum }_{i=1}^n{x}_i^2}{2{\sigma }^2}\right\}}}\\ & =g(S(\mathbf{X}),\mu )\cdot 1.\end{array}$$

Note: Dimension of sufficient statistics

In the last case, the sufficient statistic may not be one-dimensional.

Related Notes

• Sufficiency
• UMVUE