Type Alias IndexOptions

IndexOptions: {
    batchSize?: number;
    cleanup?: CleanupMode;
    cleanupBatchSize?: number;
    forceUpdate?: boolean;
    sourceIdKey?: StringOrDocFunc;
}

Type declaration

  • OptionalbatchSize?: number

    The number of documents to index in one batch.

  • Optionalcleanup?: CleanupMode

    The cleanup mode to use. Can be "full", "incremental" or undefined.

    • Incremental: Cleans up all documents that haven't been updated AND that are associated with source ids that were seen during indexing. Clean up is done continuously during indexing helping to minimize the probability of users seeing duplicated content.
    • Full: Delete all documents that haven to been returned by the loader. Clean up runs after all documents have been indexed. This means that users may see duplicated content during indexing.
    • undefined: Do not delete any documents.
  • OptionalcleanupBatchSize?: number

    Batch size to use when cleaning up documents.

  • OptionalforceUpdate?: boolean

    Force update documents even if they are present in the record manager. Useful if you are re-indexing with updated embeddings.

  • OptionalsourceIdKey?: StringOrDocFunc

    Optional key that helps identify the original source of the document. Must either be a string representing the key of the source in the metadata or a function that takes a document and returns a string representing the source. Required when cleanup is incremental.

Settings

Member Visibility