EvaluationPolicy

gepa.strategies.eval_policy.EvaluationPolicy

Bases: Protocol[DataId, DataInst]

Strategy for choosing validation ids to evaluate and identifying best programs for validation instances.

Functions

get_eval_batch(loader: DataLoader[DataId, DataInst], state: GEPAState, target_program_idx: ProgramIdx | None = None) -> list[DataId] abstractmethod

Select examples for evaluation for a program

Source code in gepa/strategies/eval_policy.py

@abstractmethod
def get_eval_batch(
    self, loader: DataLoader[DataId, DataInst], state: GEPAState, target_program_idx: ProgramIdx | None = None
) -> list[DataId]:
    """Select examples for evaluation for a program"""
    ...

get_best_program(state: GEPAState) -> ProgramIdx abstractmethod

Return "best" program given all validation results so far across candidates

Source code in gepa/strategies/eval_policy.py

@abstractmethod
def get_best_program(self, state: GEPAState) -> ProgramIdx:
    """Return "best" program given all validation results so far across candidates"""
    ...

get_valset_score(program_idx: ProgramIdx, state: GEPAState) -> float abstractmethod

Return the score of the program on the valset

Source code in gepa/strategies/eval_policy.py

@abstractmethod
def get_valset_score(self, program_idx: ProgramIdx, state: GEPAState) -> float:
    """Return the score of the program on the valset"""
    ...