How to save/use Microsoft.ML's cached bottleneck computation values

126 views Asked by At

When training a model using Microsoft.ML the bottleneck computation can take a LONG time.

Phase: Bottleneck Computation, Dataset used: Validation, Image Index:     1
Phase: Bottleneck Computation, Dataset used: Validation, Image Index:     2
Phase: Bottleneck Computation, Dataset used: Validation, Image Index:     3
...
Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 30000

In ImageClassificationTrainer.Options() there are two options, called:

ReuseValidationSetBottleneckCachedValues = true,
ReuseTrainSetBottleneckCachedValues = true,

These should ensure that when training again after you trained before the cashed bottleneck values are used to speed up the process (which would have saved me days by now). Unfortunately this is not the case as the bottleneck computations get redone every time, and I can't find anything about this..

How to use Microsoft.ML's Bottleneck computation value cashing?

2

There are 2 answers

0
Luis Quintanilla On BEST ANSWER

You can do that by specifing the WorkspacePath parameter.

0
Malinko On

As Luis Quintanilla answered the workspace needs to be specified. Just to clarify this is done in the ImageClassificationTrainer.Options class:

var options = new ImageClassificationTrainer.Options()
{
    FeatureColumnName = "Image",
    LabelColumnName = "Label",
    Arch = arch,
    Epoch = 100,
    BatchSize = 16,
    LearningRate = 0.001f,
    ReuseValidationSetBottleneckCachedValues = true,
    ReuseTrainSetBottleneckCachedValues = true,
    WorkspacePath = "./",                                             // <- like so
    MetricsCallback = (metrics) => Console.WriteLine(metrics),
    ValidationSet = testDataset
};