\( \newcommand{\R}{\mathbb R} \newcommand{\X}{\mathcal X} \newcommand{\Z}{\mathcal Z} \newcommand{\bP}{\mathbb P} \newcommand{\bQ}{\mathbb Q} \newcommand{\Preal}{{\mathbb P}_0} \newcommand{\Ptheta}{{\mathbb P}_\theta} \newcommand{\PZ}{{\mathbb P}_Z} \newcommand{\preal}{p_0} \newcommand{\ptheta}{p_\theta} \newcommand{\pZ}{p_Z} \DeclareMathOperator*{\argmin}{argmin} \DeclareMathOperator*{\argmax}{argmax} \DeclareMathOperator*{\E}{\mathbb E} \DeclareMathOperator*{\Var}{Var} \DeclareMathOperator{\JS}{JS} \DeclareMathOperator{\KL}{KL} \DeclareMathOperator{\mmd}{MMD} \DeclareMathOperator{\mmdhat}{\widehat{MMD}} \DeclareMathOperator{\dim}{dim} \DeclareMathOperator{\supp}{supp} \newcommand{\train}{\text{train}} \newcommand{\test}{\text{test}} \newcommand{\D}{\mathcal D} \newcommand{\F}{\mathcal F} \newcommand{\N}{\mathcal N} \newcommand{\h}{\mathcal H} \newcommand{\Kxy}{K_{XY}} \newcommand{\Ktxx}{\tilde K_{XX}} \newcommand{\Ktyy}{\tilde K_{YY}} \newcommand{\tp}{^\mathsf{T}} \)



Two-Sample Tests,
Integral Probability Metrics,
and GAN Objectives


Dougal J. Sutherland

Gatsby unit, University College London

GAN training example

Epoch 0

Samples from $\Ptheta$

Samples from $\Preal$

GAN training example

Epoch 1

Samples from $\Ptheta$

Samples from $\Preal$

GAN training example

Epoch 2

Samples from $\Ptheta$

Samples from $\Preal$

GAN training example

Epoch 3

Samples from $\Ptheta$

Samples from $\Preal$

GAN training example

Epoch 4

Samples from $\Ptheta$

Samples from $\Preal$

GAN training example

Epoch 5

Samples from $\Ptheta$

Samples from $\Preal$

GAN training example

Epoch 10

Samples from $\Ptheta$

Samples from $\Preal$

GAN training example

Epoch 100

Samples from $\Ptheta$

Samples from $\Preal$

GAN training example

Epoch 500

Samples from $\Ptheta$

Samples from $\Preal$

GAN training example

Epoch 900

Samples from $\Ptheta$

Samples from $\Preal$

Another GAN training example

Epoch 0

Samples from $\Ptheta$

Samples from $\Preal$

Another GAN training example

Epoch 1

Samples from $\Ptheta$

Samples from $\Preal$

Another GAN training example

Epoch 2

Samples from $\Ptheta$

Samples from $\Preal$

Another GAN training example

Epoch 3

Samples from $\Ptheta$

Samples from $\Preal$

Another GAN training example

Epoch 4

Samples from $\Ptheta$

Samples from $\Preal$

MMD witness function

MMD witness function

MMD witness function

MMD witness function

MMD witness function

MMD witness function

The kernel matters!

$k$ controls class of possible witness functions

The kernel matters!

$k$ controls class of possible witness functions

The kernel matters!

$k$ controls class of possible witness functions

Choosing a kernel

Choosing a kernel

Choosing a kernel

Investigating mistakes

Toy problem

$m = 500$