maite.workflows.evaluate#
- maite.workflows.evaluate(*, model, metric=None, dataloader=None, dataset=None, batch_size=1, augmentation=None, return_augmented_data=False, return_preds=False)[source]#
Evaluate a model’s performance on data according to some metric with optional augmentation.
Some data source (either a dataloader or a dataset) must be provided or an InvalidArgument exception is raised.
- Parameters:
- modelSomeModel
Maite Model object.
- metricSomeMetric | None, (default=None)
Compatible maite Metric.
- dataloaderSomeDataloader | None, (default=None)
Compatible maite dataloader.
- datasetSomeDataset | None, (default=None)
Compatible maite dataset.
- batch_sizeint, (default=1)
Batch size for use with dataset (ignored if dataset=None).
- augmentationSomeAugmentation | None, (default=None)
Compatible maite augmentation.
- return_augmented_databool, (default=False)
Set to True to return post-augmentation data as a function output.
- return_predsbool, (default=False)
Set to True to return raw predictions as a function output.
- Returns:
- tuple[dict[str, Any], Sequence[TargetType], Sequence[tuple[InputBatchType, TargetBatchType, DatumMetadataBatchType]]]
Tuple of returned metric value, sequence of model predictions, and sequence of data batch tuples fed to the model during inference. The actual types represented by InputBatchType, TargetBatchType, and DatumMetadataBatchType will vary by the domain of the components provided as input arguments (e.g. image classification or object detection.) Note that the second and third return arguments will be empty if return_augmented_data is False or return_preds is False, respectively.