MistralAI completions LLM.

Hierarchy (view full)

Implements

Constructors

Properties

apiKey: string

The API key to use.

{process.env.MISTRAL_API_KEY}
batchSize: number = 20

Batch size to use when passing multiple documents to generate

model: string = "codestral-latest"

The name of the model to use.

"codestral-latest"
streaming: boolean = false

Whether or not to stream the response.

{false}
temperature: number = 0

What sampling temperature to use, between 0.0 and 2.0. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic.

{0.7}
endpoint?: string

Override the default endpoint.

maxConcurrency?: number
maxRetries?: number
maxTokens?: number

The maximum number of tokens to generate in the completion. The token count of your prompt plus maxTokens cannot exceed the model's context length.

randomSeed?: number

The seed to use for random sampling. If set, different calls will generate deterministic results. Alias for seed

topP?: number

Nucleus sampling, where the model considers the results of the tokens with topP probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. Should be between 0 and 1.

{1}

Methods

  • Parameters

    • request: CompletionRequest
    • options: unknown
    • stream: false

    Returns Promise<ChatCompletionResponse>

  • Parameters

    • request: CompletionRequest
    • options: unknown
    • stream: true

    Returns Promise<AsyncGenerator<ChatCompletionResponseChunk, void, unknown>>