Class ChatAnthropicMessages<CallOptions>

Wrapper around Anthropic large language models.

To use you should have the @anthropic-ai/sdk package installed, with the ANTHROPIC_API_KEY environment variable set.

Remarks

Any parameters that are valid to be passed to anthropic.beta.messages can be passed through invocationKwargs, even if not explicitly available on this class.

Example

const model = new ChatAnthropic({
temperature: 0.9,
anthropicApiKey: 'YOUR-API-KEY',
});
const res = await model.invoke({ input: 'Hello!' });
console.log(res);

Type Parameters

Hierarchy

Implements

Constructors

Properties

ParsedCallOptions: Omit<CallOptions, never>
caller: AsyncCaller

The async caller should be used by subclasses to make any async calls, which will thus benefit from the concurrency and retry logic.

clientOptions: ClientOptions

Overridable Anthropic ClientOptions

maxTokens: number = 2048

A maximum number of tokens to generate before stopping.

modelName: string = "claude-2.1"

Model name to use

streaming: boolean = false

Whether to stream the results or not

temperature: number = 1

Amount of randomness injected into the response. Ranges from 0 to 1. Use temp closer to 0 for analytical / multiple choice, and temp closer to 1 for creative and generative tasks.

topK: number = -1

Only sample from the top K options for each subsequent token. Used to remove "long tail" low probability responses. Defaults to -1, which disables it.

topP: number = -1

Does nucleus sampling, in which we compute the cumulative distribution over all the options for each subsequent token in decreasing probability order and cut it off once it reaches a particular probability specified by top_p. Defaults to -1, which disables it. Note that you should either alter temperature or top_p, but not both.

verbose: boolean

Whether to print out response text.

anthropicApiKey?: string

Anthropic API key

apiUrl?: string
callbacks?: Callbacks
invocationKwargs?: Kwargs

Holds any additional parameters that are valid to pass to anthropic.complete that are not explicitly specified on this class.

metadata?: Record<string, unknown>
name?: string
stopSequences?: string[]

A list of strings upon which to stop generating. You probably want ["\n\nHuman:"], as that's the cue for the next turn in the dialog agent.

tags?: string[]
batchClient: Anthropic
streamingClient: Anthropic

Accessors

  • get callKeys(): string[]
  • Keys that the language model accepts as call options.

    Returns string[]

Methods

  • Assigns new fields to the dict output of this runnable. Returns a new runnable.

    Parameters

    • mapping: RunnableMapLike<Record<string, unknown>, Record<string, unknown>>

    Returns RunnableSequence<any, any>

  • Default implementation of batch, which calls invoke N times. Subclasses should override this method if they can batch more efficiently.

    Parameters

    • inputs: BaseLanguageModelInput[]

      Array of inputs to each batch call.

    • Optional options: Partial<CallOptions> | Partial<CallOptions>[]

      Either a single call options object to apply to each batch call or an array for each call.

    • Optional batchOptions: RunnableBatchOptions & {
          returnExceptions?: false;
      }

    Returns Promise<BaseMessageChunk[]>

    An array of RunOutputs, or mixed RunOutputs and errors if batchOptions.returnExceptions is set

  • Parameters

    Returns Promise<(Error | BaseMessageChunk)[]>

  • Parameters

    Returns Promise<(Error | BaseMessageChunk)[]>

  • Parameters

    • messages: BaseMessageLike[]

      An array of BaseMessage instances.

    • Optional options: string[] | CallOptions

      The call options or an array of stop sequences.

    • Optional callbacks: Callbacks

      The callbacks for the language model.

    Returns Promise<BaseMessage>

    A Promise that resolves to a BaseMessage.

    ⚠️ Deprecated ⚠️

    Use .invoke() instead. Will be removed in 0.2.0.

    This feature is deprecated and will be removed in the future.

    It is not recommended for use.

    Makes a single call to the chat model.

  • Parameters

    • promptValue: BasePromptValueInterface

      The value of the prompt.

    • Optional options: string[] | CallOptions

      The call options or an array of stop sequences.

    • Optional callbacks: Callbacks

      The callbacks for the language model.

    Returns Promise<BaseMessage>

    A Promise that resolves to a BaseMessage.

    Deprecated

    Use .invoke() instead. Will be removed in 0.2.0.

    Makes a single call to the chat model with a prompt value.

  • Generates chat based on the input messages.

    Parameters

    • messages: BaseMessageLike[][]

      An array of arrays of BaseMessage instances.

    • Optional options: string[] | CallOptions

      The call options or an array of stop sequences.

    • Optional callbacks: Callbacks

      The callbacks for the language model.

    Returns Promise<LLMResult>

    A Promise that resolves to an LLMResult.

  • Generates a prompt based on the input prompt values.

    Parameters

    • promptValues: BasePromptValueInterface[]

      An array of BasePromptValue instances.

    • Optional options: string[] | CallOptions

      The call options or an array of stop sequences.

    • Optional callbacks: Callbacks

      The callbacks for the language model.

    Returns Promise<LLMResult>

    A Promise that resolves to an LLMResult.

  • Parameters

    • Optional suffix: string

    Returns string

  • Parameters

    Returns Promise<number>

  • Get the parameters used to invoke the model

    Parameters

    • Optional options: Omit<CallOptions, never>

    Returns Omit<AnthropicMessageCreateParams | AnthropicStreamingMessageCreateParams, "messages" | "anthropic-beta"> & Kwargs

  • Invokes the chat model with a single input.

    Parameters

    • input: BaseLanguageModelInput

      The input for the language model.

    • Optional options: CallOptions

      The call options.

    Returns Promise<BaseMessageChunk>

    A Promise that resolves to a BaseMessageChunk.

  • Pick keys from the dict output of this runnable. Returns a new runnable.

    Parameters

    • keys: string | string[]

    Returns RunnableSequence<any, any>

  • Create a new runnable sequence that runs each individual runnable in series, piping the output of one runnable into another runnable or runnable-like.

    Type Parameters

    • NewRunOutput

    Parameters

    Returns RunnableSequence<BaseLanguageModelInput, Exclude<NewRunOutput, Error>>

    A new runnable sequence.

  • Parameters

    • text: string

      The text input.

    • Optional options: string[] | CallOptions

      The call options or an array of stop sequences.

    • Optional callbacks: Callbacks

      The callbacks for the language model.

    Returns Promise<string>

    A Promise that resolves to a string.

    Deprecated

    Use .invoke() instead. Will be removed in 0.2.0.

    Predicts the next message based on a text input.

  • Parameters

    • messages: BaseMessage[]

      An array of BaseMessage instances.

    • Optional options: string[] | CallOptions

      The call options or an array of stop sequences.

    • Optional callbacks: Callbacks

      The callbacks for the language model.

    Returns Promise<BaseMessage>

    A Promise that resolves to a BaseMessage.

    Deprecated

    Use .invoke() instead. Will be removed in 0.2.0.

    Predicts the next message based on the input messages.

  • Stream output in chunks.

    Parameters

    Returns Promise<IterableReadableStream<BaseMessageChunk>>

    A readable stream that is also an iterable.

  • Stream all output from a runnable, as reported to the callback system. This includes all inner runs of LLMs, Retrievers, Tools, etc. Output is streamed as Log objects, which include a list of jsonpatch ops that describe how the state of the run has changed in each step, and the final state of the run. The jsonpatch ops can be applied in order to construct state.

    Parameters

    • input: BaseLanguageModelInput
    • Optional options: Partial<CallOptions>
    • Optional streamOptions: Omit<LogStreamCallbackHandlerInput, "autoClose">

    Returns AsyncGenerator<RunLogPatch, any, unknown>

  • Default implementation of transform, which buffers input and then calls stream. Subclasses should override this method if they can start producing output while input is still being generated.

    Parameters

    Returns AsyncGenerator<BaseMessageChunk, any, unknown>

  • Bind lifecycle listeners to a Runnable, returning a new Runnable. The Run object contains information about the run, including its id, type, input, output, error, startTime, endTime, and any tags or metadata added to the run.

    Parameters

    • params: {
          onEnd?: ((run, config?) => void | Promise<void>);
          onError?: ((run, config?) => void | Promise<void>);
          onStart?: ((run, config?) => void | Promise<void>);
      }

      The object containing the callback functions.

      • Optional onEnd?: ((run, config?) => void | Promise<void>)
          • (run, config?): void | Promise<void>
          • Called after the runnable finishes running, with the Run object.

            Parameters

            Returns void | Promise<void>

      • Optional onError?: ((run, config?) => void | Promise<void>)
          • (run, config?): void | Promise<void>
          • Called if the runnable throws an error, with the Run object.

            Parameters

            Returns void | Promise<void>

      • Optional onStart?: ((run, config?) => void | Promise<void>)
          • (run, config?): void | Promise<void>
          • Called before the runnable starts running, with the Run object.

            Parameters

            Returns void | Promise<void>

    Returns Runnable<BaseLanguageModelInput, BaseMessageChunk, CallOptions>

  • Creates a streaming request with retry.

    Parameters

    • request: AnthropicStreamingMessageCreateParams & Kwargs

      The parameters for creating a completion.

    Returns Promise<Stream<MessageStreamEvent>>

    A streaming request.

  • Formats messages as a prompt for the model.

    Parameters

    • messages: BaseMessage[]

      The base messages to format as a prompt.

    Returns {
        messages: MessageParam[];
        system?: string;
    }

    The formatted prompt.

    • messages: MessageParam[]
    • Optional system?: string

Generated using TypeDoc