torch_mimicry.metrics

FID

PyTorch interface for computing FID.

compute_gen_dist_stats(netG, num_samples, sess, device, seed, batch_size, print_every=20, verbose=True)[source]

Directly produces the images and convert them into numpy format without saving the images on disk.

Parameters:
  • netG (Module) – Torch Module object representing the generator model.
  • num_samples (int) – The number of fake images for computing statistics.
  • sess (Session) – TensorFlow session to use.
  • device (str) – Device identifier to use for computation.
  • seed (int) – The random seed to use.
  • batch_size (int) – The number of samples per batch for inference.
  • print_every (int) – Interval for printing log.
  • verbose (bool) – If True, prints progress.
Returns:

Mean features stored as np array. ndarray: Covariance of features stored as np array.
Return type:

ndarray
compute_real_dist_stats(num_samples, sess, batch_size, dataset=None, stats_file=None, seed=0, verbose=True, log_dir='./log')[source]

Reads the image data and compute the FID mean and cov statistics for real images.

Parameters:
  • num_samples (int) – Number of real images to compute statistics.
  • sess (Session) – TensorFlow session to use.
  • dataset (str/Dataset) – Dataset to load.
  • batch_size (int) – The batch size to feedforward for inference.
  • stats_file (str) – The statistics file to load from if there is already one.
  • verbose (bool) – If True, prints progress of computation.
  • log_dir (str) – Directory where feature statistics can be stored.
Returns:

Mean features stored as np array. ndarray: Covariance of features stored as np array.
Return type:

ndarray
fid_score(num_real_samples, num_fake_samples, netG, dataset, seed=0, device=None, batch_size=50, verbose=True, stats_file=None, log_dir='./log')[source]

Computes FID stats using functions that store images in memory for speed and fidelity. Fidelity since by storing images in memory, we don’t subject the scores to different read/write implementations of imaging libraries.

Parameters:
  • num_real_samples (int) – The number of real images to use for FID.
  • num_fake_samples (int) – The number of fake images to use for FID.
  • netG (Module) – Torch Module object representing the generator model.
  • device (str/torch.device) – Device identifier to use for computation.
  • seed (int) – The random seed to use.
  • dataset (str/Dataset) – The name of the dataset to load if known, or a custom Dataset object
  • batch_size (int) – The batch size to feedforward for inference.
  • verbose (bool) – If True, prints progress.
  • stats_file (str) – The statistics file to load from if there is already one.
  • log_dir (str) – Directory where feature statistics can be stored.
Returns:

Scalar FID score.
Return type:

float

KID

PyTorch interface for computing KID.

compute_gen_dist_feat(netG, num_samples, sess, device, seed, batch_size, print_every=20, verbose=True)[source]

Directly produces the images and convert them into numpy format without saving the images on disk.

Parameters:
  • netG (Module) – Torch Module object representing the generator model.
  • num_samples (int) – The number of fake images for computing features.
  • sess (Session) – TensorFlow session to use.
  • device (str) – Device identifier to use for computation.
  • seed (int) – The random seed to use.
  • batch_size (int) – The number of samples per batch for inference.
  • print_every (int) – Interval for printing log.
  • verbose (bool) – If True, prints progress.
Returns:

Inception features of generated images.
Return type:

ndarray
compute_real_dist_feat(num_samples, sess, dataset, batch_size, seed=0, verbose=True, feat_file=None, log_dir='./log')[source]

Reads the image data and compute the real image features.

Parameters:
  • num_samples (int) – Number of real images to compute features.
  • sess (Session) – TensorFlow session to use.
  • dataset (str/Dataset) – Dataset to load.
  • batch_size (int) – The batch size to feedforward for inference.
  • feat_file (str) – The features file to load from if there is already one.
  • verbose (bool) – If True, prints progress of computation.
  • log_dir (str) – Directory where features can be stored.
Returns:

Inception features of real images.
Return type:

ndarray
kid_score(num_samples, netG, dataset, seed=0, device=None, num_subsets=10, batch_size=50, verbose=True, feat_file=None, log_dir='./log')[source]

Computes KID score.

Parameters:
  • num_samples (int) – The number of real and fake images to use for KID.
  • num_subsets (int) – Number of subsets to compute average MMD.
  • netG (Module) – Torch Module object representing the generator model.
  • device (str) – Device identifier to use for computation.
  • seed (int) – The random seed to use.
  • dataset (str/Dataset) – The name of the dataset to load if known, or a custom Dataset object
  • batch_size (int) – The batch size to feedforward for inference.
  • feat_file (str) – The path to specific inception features for real images.
  • log_dir (str) – Directory where features can be stored.
  • verbose (bool) – If True, prints progress.
Returns:

Scalar mean and std of KID scores computed.
Return type:

tuple

Inception Score

PyTorch interface for computing Inception Score.

inception_score(num_samples, netG, device=None, batch_size=50, splits=10, log_dir='./log', seed=0, print_every=20)[source]

Computes the inception score of generated images.

Parameters:
  • netG (Module) – The generator model to use for generating images.
  • device (str/torch.device) – Device identifier to use for computation.
  • num_samples (int) – The number of samples to generate.
  • batch_size (int) – Batch size per feedforward step for inception model.
  • splits (int) – The number of splits to use for computing IS.
  • log_dir (str) – Path to store metric computation objects.
  • seed (int) – Random seed for generation.
Returns:

Mean and standard deviation of the inception score computed from using num_samples generated images.

Metrics

Computes different GAN metrics for a generator.

evaluate(metric, netG, log_dir, evaluate_range=None, evaluate_step=None, num_runs=3, start_seed=0, overwrite=False, write_to_json=True, device=None, **kwargs)[source]

Evaluates a generator over several runs.

Parameters:
  • metric (str) – The name of the metric for evaluation.
  • netG (Module) – Torch generator model to evaluate.
  • log_dir (str) – The path to the log directory.
  • evaluate_range (tuple) – The 3 valued tuple for defining a for loop.
  • evaluate_step (int) – The specific checkpoint to load. Used in place of evaluate_range.
  • device (str) – Device identifier to use for computation.
  • num_runs (int) – The number of runs to compute FID for each checkpoint.
  • start_seed (int) – Starting random seed to use.
  • write_to_json (bool) – If True, writes to an output json file in log_dir.
  • overwrite (bool) – If True, then overwrites previous metric score.
Returns:

None