Evaluation

Evaluators are callables: construct one with examples, call it with a model, and read back a metric dictionary. The Training Overview shows where they fit in a training run.

BinaryClassificationEvaluator

class relational_transformers.BinaryClassificationEvaluator(examples, threshold=0.5, task_head=None)

Evaluate binary predictions at a configurable probability threshold.

Parameters:

examples (Sequence[RelationalExample])
threshold (float)
task_head (str | None)

RegressionEvaluator

class relational_transformers.RegressionEvaluator(examples, task_head=None)

Compute MAE, RMSE, and R² for regression or forecasting predictions.

Parameters:

examples (Sequence[RelationalExample])
task_head (str | None)

AblationEvaluator

class relational_transformers.AblationEvaluator(examples, ablations)

Measure prediction deltas for caller-defined groups of cell positions.

Parameters:

examples (Sequence[RelationalExample])
ablations (Mapping[str, Sequence[int]])

SequentialEvaluator

class relational_transformers.SequentialEvaluator(evaluators)

Run multiple evaluators and merge their non-overlapping metrics.

Parameters:: evaluators (Sequence)