org.grouplens.lenskit.eval.traintest

Class SimpleEvaluator

java.lang.Object

org.grouplens.lenskit.eval.traintest.SimpleEvaluator

All Implemented Interfaces:

Callable<Table>

public class SimpleEvaluator
extends Object
implements Callable<Table>

Constructor Summary

Constructors

Constructor and Description
SimpleEvaluator() Construct a simple evaluator.
SimpleEvaluator(Properties props) Create a simple evaluator with a custom configuration.

Method Summary

Modifier and Type	Method and Description
SimpleEvaluator	addAlgorithm(AlgorithmInstance algo) Adds an algorithmInfo to the TrainTestEvalCommand being built.
SimpleEvaluator	addAlgorithm(AlgorithmInstanceBuilder algo) Adds a fully configured algorithmInfo command to the TrainTestEvalCommand being built.
SimpleEvaluator	addAlgorithm(String name, LenskitConfiguration config) An algorithm instance constructed with a name and Lenskit configuration
SimpleEvaluator	addDataset(CrossfoldTask cross) Calls the CrossfoldCommand and adds the resulting TTDataSets to the TrainTestEvalCommand.
SimpleEvaluator	addDataset(DataSource train, DataSource test) This constructs a new TTDataSet and passes it to the TrainTestEvalCommand.
SimpleEvaluator	addDataset(DataSource source, int partitions) Add a new data set to be cross-folded.
SimpleEvaluator	addDataset(DataSource source, int partitions, double holdout) Add a new data set to be cross-folded.
SimpleEvaluator	addDataset(String name, DataSource source, int partitions) Add a new data set to be cross-folded.
SimpleEvaluator	addDataset(String name, DataSource source, int partitions, double holdout) Add a new data set to be cross-folded.
SimpleEvaluator	addDataset(String name, EventDAO train, EventDAO test, PreferenceDomain dom) This constructs a new TTDataSet and passes it to the TrainTestEvalCommand.
SimpleEvaluator	addDataset(TTDataSet data) Adds a single TTDataSet to the TrainTestEvalCommand.
<T> SimpleEvaluator	addMetric(Class<? extends Metric<T>> metric) Adds a completed metric to the TrainTestEvalCommand
SimpleEvaluator	addMetric(Metric<?> metric) Adds a completed metric to the TrainTestEvalCommand
Table	call() If this is called more than once it will call of these commands again and most likely throw an exception.
EvalConfig	getEvalConfig()
TrainTestEvalTask	getRawCommand() Provides raw unrestricted access for the command.
SimpleEvaluator	setOutput(File file) This provides a wrapper around TrainTestEvalCommand.setOutput()
SimpleEvaluator	setOutputPath(String path) Creates a new file with the name and passes it to TrainTestEvalCommand.setOutput()
SimpleEvaluator	setPredictOutput(File file) This provides a wrapper around TrainTestEvalCommand.setPredictOutput
SimpleEvaluator	setPredictOutputPath(String path) Creates a new file with the name and passes it to TrainTestEvalCommand.setPredictOutput()
SimpleEvaluator	setUserOutput(File file) This provides a wrapper around TrainTestEvalCommand.setUserOutput
SimpleEvaluator	setUserOutputPath(String path) Creates a new file with the name and passes it to TrainTestEvalCommand.setUserOutput()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SimpleEvaluator

public SimpleEvaluator()

Construct a simple evaluator.

SimpleEvaluator

public SimpleEvaluator(Properties props)

Create a simple evaluator with a custom configuration.

Parameters:

props - Properties for the eval configuration.

Method Detail

getEvalConfig

public EvalConfig getEvalConfig()

addAlgorithm

public SimpleEvaluator addAlgorithm(AlgorithmInstance algo)

Adds an algorithmInfo to the TrainTestEvalCommand being built. If any exception is thrown while the command is called it is rethrown as a runtime error.

Parameters:

algo - The algorithmInfo added to the TrainTestEvalCommand

Returns:

Itself to allow chaining

addAlgorithm

public SimpleEvaluator addAlgorithm(String name,
                                    LenskitConfiguration config)

An algorithm instance constructed with a name and Lenskit configuration

Parameters:

name -

config - Lenskit configuration

addAlgorithm

public SimpleEvaluator addAlgorithm(AlgorithmInstanceBuilder algo)

Adds a fully configured algorithmInfo command to the TrainTestEvalCommand being built.

Parameters:

algo - The algorithmInfo added to the TrainTestEvalCommand

Returns:

Itself to allow chaining

addDataset

public SimpleEvaluator addDataset(CrossfoldTask cross)

Calls the CrossfoldCommand and adds the resulting TTDataSets to the TrainTestEvalCommand. Any exceptions that are thrown are wrapped as RuntimeExceptions.

Parameters:

cross -

Returns:

Itself to allow for method chaining.

addDataset

public SimpleEvaluator addDataset(String name,
                                  DataSource source,
                                  int partitions,
                                  double holdout)

Add a new data set to be cross-folded. This method creates a new CrossfoldTask and passes it to addDataset(CrossfoldTask). All crossfold parameters that are not taken as arguments by this method are left at their defaults.

Parameters:

name - The name of the crossfold

source - The source for the crossfold

partitions - The number of partitions

holdout - The holdout fraction

Returns:

Itself for chaining.

addDataset

public SimpleEvaluator addDataset(DataSource source,
                                  int partitions,
                                  double holdout)

Add a new data set to be cross-folded. This method creates a new CrossfoldTask and passes it to addDataset(CrossfoldTask). All crossfold parameters that are not taken as arguments by this method are left at their defaults.

Parameters:

source - The source for the crossfold

partitions - The number of partitions

holdout - The holdout fraction

Returns:

Itself for chaining.

addDataset

public SimpleEvaluator addDataset(String name,
                                  DataSource source,
                                  int partitions)

Add a new data set to be cross-folded. This method creates a new CrossfoldTask and passes it to addDataset(CrossfoldTask). All crossfold parameters that are not taken as arguments by this method are left at their defaults.

Note: Prior to LensKit 2.2, this method used a holdout fraction of 0.2. In LensKit 2.2, it was changed to use the CrossfoldTask's default holdout.

Parameters:

name - The name of the crossfold

source - The source for the crossfold

partitions - The number of partitions

Returns:

Itself for chaining.

addDataset

public SimpleEvaluator addDataset(DataSource source,
                                  int partitions)

Add a new data set to be cross-folded. This method creates a new CrossfoldTask and passes it to addDataset(CrossfoldTask). All crossfold parameters that are not taken as arguments by this method are left at their defaults.

Note: Prior to LensKit 2.2, this method used a holdout fraction of 0.2. In LensKit 2.2, it was changed to use the CrossfoldTask's default holdout.

Parameters:

source - The source for the crossfold

partitions - The number of partitions

Returns:

Itself for chaining.

addDataset

public SimpleEvaluator addDataset(TTDataSet data)

Adds a single TTDataSet to the TrainTestEvalCommand. This acts a wrapper around TrainTestEvalCommand.addDataset

Parameters:

data - The dataset to be added to the command.

Returns:

Itself to allow for method chaining.

addDataset

public SimpleEvaluator addDataset(String name,
                                  EventDAO train,
                                  EventDAO test,
                                  PreferenceDomain dom)

This constructs a new TTDataSet and passes it to the TrainTestEvalCommand.

Parameters:

name - The name of the new dataset.

train - The DAOFactory with the train data.

test - The DAOFactory with the test data.

dom - The PreferenceDomain to be supplied to the new TTDataSet

Returns:

Itself for method chaining.

addDataset

public SimpleEvaluator addDataset(DataSource train,
                                  DataSource test)

This constructs a new TTDataSet and passes it to the TrainTestEvalCommand. The name for the data source will default to 'generic-data-source'. Because of this, be careful of calling this method more than once.

Parameters:

train - The DAOFactory with the train data.

test - The DAOFactory with the test data.

Returns:

Itself for method chaining.

addMetric

public SimpleEvaluator addMetric(Metric<?> metric)

Adds a completed metric to the TrainTestEvalCommand

Parameters:

metric - The metric to be added.

Returns:

Itself for method chaining.

addMetric

public <T> SimpleEvaluator addMetric(Class<? extends Metric<T>> metric)

Adds a completed metric to the TrainTestEvalCommand

Parameters:

metric - The metric to be added.

Returns:

Itself for method chaining.

setOutput

public SimpleEvaluator setOutput(File file)

This provides a wrapper around TrainTestEvalCommand.setOutput()

Parameters:

file - The file set as the output of the command

Returns:

Itself for method chaining

setPredictOutput

public SimpleEvaluator setPredictOutput(File file)

This provides a wrapper around TrainTestEvalCommand.setPredictOutput

Parameters:

file - The file set as the prediction output.

Returns:

setUserOutput

public SimpleEvaluator setUserOutput(File file)

This provides a wrapper around TrainTestEvalCommand.setUserOutput

Parameters:

file - The file set as the prediction user.

Returns:

setOutputPath

public SimpleEvaluator setOutputPath(String path)

Creates a new file with the name and passes it to TrainTestEvalCommand.setOutput()

Parameters:

path - The path to the file to be created

Returns:

Itself for method chaining

setPredictOutputPath

public SimpleEvaluator setPredictOutputPath(String path)

Creates a new file with the name and passes it to TrainTestEvalCommand.setPredictOutput()

Parameters:

path - The path to the file to be created

Returns:

Itself for method chaining

setUserOutputPath

public SimpleEvaluator setUserOutputPath(String path)

Creates a new file with the name and passes it to TrainTestEvalCommand.setUserOutput()

Parameters:

path - The path to the file to be created

Returns:

Itself for method chaining

getRawCommand

public TrainTestEvalTask getRawCommand()

Provides raw unrestricted access for the command. Use this with caution! Calling certain methods on the TrainTestDataSet can force this command to throw an exception farther down the line.

Returns:

The raw partially configured command.

call

public Table call()
           throws TaskExecutionException

If this is called more than once it will call of these commands again and most likely throw an exception.

Specified by:

call in interface Callable<Table>

Returns:

The table resulting from calling the command.

Throws:

TaskExecutionException