org.lenskit.eval.traintest.predict

Class NDCGPredictMetric

java.lang.Object

org.lenskit.eval.traintest.metrics.Metric<X>

org.lenskit.eval.traintest.predict.PredictMetric<org.apache.commons.math3.stat.descriptive.moment.Mean>

org.lenskit.eval.traintest.predict.NDCGPredictMetric

public class NDCGPredictMetric
extends PredictMetric<org.apache.commons.math3.stat.descriptive.moment.Mean>

Evaluate a recommender’s predictions with normalized discounted cumulative gain.

This is a prediction evaluator that uses base-2 nDCG to evaluate recommender accuracy. The items are ordered by predicted preference and the nDCG is computed using the user’s real rating as the gain for each item. Doing this only over the queried items, rather than in the general recommend condition, avoids penalizing recommenders for recommending items that would be better if the user had known about them and provided ratings (e.g., for doing their job).

nDCG is computed per-user and then averaged over all users.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	NDCGPredictMetric.Spec Specification for configuring nDCG metrics.

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	DEFAULT_COLUMN

Constructor Summary

Constructors

Constructor and Description
NDCGPredictMetric() Create a new log_2 nDCG metric with column name “Predict.nDCG”.
NDCGPredictMetric(Discount disc) Create a new nDCG metric with column name “Predict.nDCG”.
NDCGPredictMetric(Discount disc, java.lang.String name) Create a new nDCG metric.
NDCGPredictMetric(NDCGPredictMetric.Spec spec) Construct a predict metric from a spec.

Method Summary

Modifier and Type	Method and Description
org.apache.commons.math3.stat.descriptive.moment.Mean	createContext(AlgorithmInstance algorithm, DataSet dataSet, RecommenderEngine engine) Create the context for an experimental condition (algorithm/data set pair).
MetricResult	getAggregateMeasurements(org.apache.commons.math3.stat.descriptive.moment.Mean context) Get the aggregate results from an accumulator.
MetricResult	measureUser(TestUser user, ResultMap predictions, org.apache.commons.math3.stat.descriptive.moment.Mean context) Measure a single result.

Methods inherited from class org.lenskit.eval.traintest.metrics.Metric

getAggregateColumnLabels, getColumnLabels, getRequiredRoots

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_COLUMN

public static final java.lang.String DEFAULT_COLUMN

See Also:

Constant Field Values

Constructor Detail

NDCGPredictMetric

public NDCGPredictMetric()

Create a new log_2 nDCG metric with column name “Predict.nDCG”.

NDCGPredictMetric

public NDCGPredictMetric(Discount disc)

Create a new nDCG metric with column name “Predict.nDCG”.

Parameters:

disc - The discount.

NDCGPredictMetric

public NDCGPredictMetric(NDCGPredictMetric.Spec spec)

Construct a predict metric from a spec.

Parameters:

spec - The metric spec.

NDCGPredictMetric

public NDCGPredictMetric(Discount disc,
                         java.lang.String name)

Create a new nDCG metric.

Parameters:

disc - The discount.

name - The column name.

Method Detail

createContext

@Nullable
public org.apache.commons.math3.stat.descriptive.moment.Mean createContext(AlgorithmInstance algorithm,
                                                                                     DataSet dataSet,
                                                                                     RecommenderEngine engine)

Description copied from class: Metric

Create the context for an experimental condition (algorithm/data set pair). The default implementation returns null.

Note: Contexts must be thread-safe, in that multiple concurrent calls to the appropriate user-measurement function with the same context must be safe. This can be handled either by the context itself, or by the user-measurement function.

Overrides:

createContext in class Metric<org.apache.commons.math3.stat.descriptive.moment.Mean>

Parameters:

algorithm - The algorithm.

dataSet - The data set.

engine - The LensKit recommender engine, if applicable. This can be null for an external algorithm that does not provide a LensKit recommender.

Returns:

The context. This will be passed to the individual measurement methods. If the metric does need to accumulate any results, this method can return null.

getAggregateMeasurements

@Nonnull
public MetricResult getAggregateMeasurements(org.apache.commons.math3.stat.descriptive.moment.Mean context)

Description copied from class: Metric

Get the aggregate results from an accumulator. The default implementation returns MetricResult.empty().

Overrides:

getAggregateMeasurements in class Metric<org.apache.commons.math3.stat.descriptive.moment.Mean>

Parameters:

context - The context for an experimental condition.

Returns:

The aggregate results from the accumulator.

measureUser

@Nonnull
public MetricResult measureUser(TestUser user,
                                         ResultMap predictions,
                                         org.apache.commons.math3.stat.descriptive.moment.Mean context)

Description copied from class: PredictMetric

Measure a single result. The result may come from either prediction or recommendation.

Thread Safety: This method may be called concurrently by multiple threads with the same recommender and context.

Specified by:

measureUser in class PredictMetric<org.apache.commons.math3.stat.descriptive.moment.Mean>

Parameters:

user - The user’s test data.

predictions - The predictions.

Returns:

A list of fields to add to the result’s output.