Use embedding distances to score semantic difference between two predictions.

Hierarchy

  • PairwiseStringEvaluator
    • PairwiseEmbeddingDistanceEvalChain

Implements

Constructors

Properties

distanceMetric: EmbeddingDistanceType = "cosine"

The distance metric to use for comparing the embeddings.

outputKey: string = "score"
requiresInput: boolean = false
requiresReference: boolean = false
embedding?: any

The embedding objects to vectorize the outputs.

evaluationName?: string = ...

The name of the evaluation.

memory?: any
skipInputWarning?: string = ...
skipReferenceWarning?: string = ...

Accessors

Methods

  • Parameters

    • inputs: ChainValues[]
    • Optionalconfig: any[]

    Returns Promise<ChainValues[]>

    Use .batch() instead. Will be removed in 0.2.0.

    Call the chain on all inputs in the list

  • Parameters

    • values: any
    • Optionalconfig: any
    • Optionaltags: string[]

    Returns Promise<ChainValues>

    Use .invoke() instead. Will be removed in 0.2.0.

    Run the core logic of this chain and add to output if desired.

    Wraps _call and handles memory.

  • Check if the evaluation arguments are valid.

    Parameters

    • Optionalreference: string

      The reference label.

    • Optionalinput: string

      The input string.

    Returns void

    If the evaluator requires an input string but none is provided, or if the evaluator requires a reference label but none is provided.

  • Evaluate the output string pairs.

    Parameters

    • args: PairwiseStringEvaluatorArgs
    • Optionalconfig: any

    Returns Promise<ChainValues>

    A dictionary containing the preference, scores, and/or other information.

  • Invoke the chain with the provided input and returns the output.

    Parameters

    • input: ChainValues

      Input values for the chain run.

    • Optionaloptions: any

    Returns Promise<ChainValues>

    Promise that resolves with the output of the chain run.

  • Parameters

    • inputs: Record<string, unknown>
    • outputs: Record<string, unknown>
    • returnOnlyOutputs: boolean = false

    Returns Promise<Record<string, unknown>>

  • Parameters

    • input: any
    • Optionalconfig: any

    Returns Promise<string>

    Use .invoke() instead. Will be removed in 0.2.0.