torch_mimicry.metrics¶
Contents
FID¶
PyTorch interface for computing FID.
-
compute_gen_dist_stats
(netG, num_samples, sess, device, seed, batch_size, print_every=20, verbose=True)[source]¶ Directly produces the images and convert them into numpy format without saving the images on disk.
Parameters: - netG (Module) – Torch Module object representing the generator model.
- num_samples (int) – The number of fake images for computing statistics.
- sess (Session) – TensorFlow session to use.
- device (str) – Device identifier to use for computation.
- seed (int) – The random seed to use.
- batch_size (int) – The number of samples per batch for inference.
- print_every (int) – Interval for printing log.
- verbose (bool) – If True, prints progress.
Returns: Mean features stored as np array. ndarray: Covariance of features stored as np array.
Return type: ndarray
-
compute_real_dist_stats
(num_samples, sess, batch_size, dataset=None, stats_file=None, seed=0, verbose=True, log_dir='./log')[source]¶ Reads the image data and compute the FID mean and cov statistics for real images.
Parameters: - num_samples (int) – Number of real images to compute statistics.
- sess (Session) – TensorFlow session to use.
- dataset (str/Dataset) – Dataset to load.
- batch_size (int) – The batch size to feedforward for inference.
- stats_file (str) – The statistics file to load from if there is already one.
- verbose (bool) – If True, prints progress of computation.
- log_dir (str) – Directory where feature statistics can be stored.
Returns: Mean features stored as np array. ndarray: Covariance of features stored as np array.
Return type: ndarray
-
fid_score
(num_real_samples, num_fake_samples, netG, dataset, seed=0, device=None, batch_size=50, verbose=True, stats_file=None, log_dir='./log')[source]¶ Computes FID stats using functions that store images in memory for speed and fidelity. Fidelity since by storing images in memory, we don’t subject the scores to different read/write implementations of imaging libraries.
Parameters: - num_real_samples (int) – The number of real images to use for FID.
- num_fake_samples (int) – The number of fake images to use for FID.
- netG (Module) – Torch Module object representing the generator model.
- device (str/torch.device) – Device identifier to use for computation.
- seed (int) – The random seed to use.
- dataset (str/Dataset) – The name of the dataset to load if known, or a custom Dataset object
- batch_size (int) – The batch size to feedforward for inference.
- verbose (bool) – If True, prints progress.
- stats_file (str) – The statistics file to load from if there is already one.
- log_dir (str) – Directory where feature statistics can be stored.
Returns: Scalar FID score.
Return type:
KID¶
PyTorch interface for computing KID.
-
compute_gen_dist_feat
(netG, num_samples, sess, device, seed, batch_size, print_every=20, verbose=True)[source]¶ Directly produces the images and convert them into numpy format without saving the images on disk.
Parameters: - netG (Module) – Torch Module object representing the generator model.
- num_samples (int) – The number of fake images for computing features.
- sess (Session) – TensorFlow session to use.
- device (str) – Device identifier to use for computation.
- seed (int) – The random seed to use.
- batch_size (int) – The number of samples per batch for inference.
- print_every (int) – Interval for printing log.
- verbose (bool) – If True, prints progress.
Returns: Inception features of generated images.
Return type: ndarray
-
compute_real_dist_feat
(num_samples, sess, dataset, batch_size, seed=0, verbose=True, feat_file=None, log_dir='./log')[source]¶ Reads the image data and compute the real image features.
Parameters: - num_samples (int) – Number of real images to compute features.
- sess (Session) – TensorFlow session to use.
- dataset (str/Dataset) – Dataset to load.
- batch_size (int) – The batch size to feedforward for inference.
- feat_file (str) – The features file to load from if there is already one.
- verbose (bool) – If True, prints progress of computation.
- log_dir (str) – Directory where features can be stored.
Returns: Inception features of real images.
Return type: ndarray
-
kid_score
(num_samples, netG, dataset, seed=0, device=None, num_subsets=10, batch_size=50, verbose=True, feat_file=None, log_dir='./log')[source]¶ Computes KID score.
Parameters: - num_samples (int) – The number of real and fake images to use for KID.
- num_subsets (int) – Number of subsets to compute average MMD.
- netG (Module) – Torch Module object representing the generator model.
- device (str) – Device identifier to use for computation.
- seed (int) – The random seed to use.
- dataset (str/Dataset) – The name of the dataset to load if known, or a custom Dataset object
- batch_size (int) – The batch size to feedforward for inference.
- feat_file (str) – The path to specific inception features for real images.
- log_dir (str) – Directory where features can be stored.
- verbose (bool) – If True, prints progress.
Returns: Scalar mean and std of KID scores computed.
Return type:
Inception Score¶
PyTorch interface for computing Inception Score.
-
inception_score
(num_samples, netG, device=None, batch_size=50, splits=10, log_dir='./log', seed=0, print_every=20)[source]¶ Computes the inception score of generated images.
Parameters: - netG (Module) – The generator model to use for generating images.
- device (str/torch.device) – Device identifier to use for computation.
- num_samples (int) – The number of samples to generate.
- batch_size (int) – Batch size per feedforward step for inception model.
- splits (int) – The number of splits to use for computing IS.
- log_dir (str) – Path to store metric computation objects.
- seed (int) – Random seed for generation.
Returns: Mean and standard deviation of the inception score computed from using num_samples generated images.
Metrics¶
Computes different GAN metrics for a generator.
-
evaluate
(metric, netG, log_dir, evaluate_range=None, evaluate_step=None, num_runs=3, start_seed=0, overwrite=False, write_to_json=True, device=None, **kwargs)[source]¶ Evaluates a generator over several runs.
Parameters: - metric (str) – The name of the metric for evaluation.
- netG (Module) – Torch generator model to evaluate.
- log_dir (str) – The path to the log directory.
- evaluate_range (tuple) – The 3 valued tuple for defining a for loop.
- evaluate_step (int) – The specific checkpoint to load. Used in place of evaluate_range.
- device (str) – Device identifier to use for computation.
- num_runs (int) – The number of runs to compute FID for each checkpoint.
- start_seed (int) – Starting random seed to use.
- write_to_json (bool) – If True, writes to an output json file in log_dir.
- overwrite (bool) – If True, then overwrites previous metric score.
Returns: None