Type Alias LlmModelMetadata

Stats about a given LLM model.

type LlmModelMetadata = {
    contextWindowTokens: number;
    displayName: string;
    maxOutputTokens: number;
}

Properties

contextWindowTokens: number

Total context window size: input (aka prompt) + output (aka completion) tokens combined.

displayName: string

A friendly name for the model, to display in UIs.

maxOutputTokens: number

Maximum tokens the model can generate in a single response.