Skip to content

llama_cpp

pixeltable.functions.llama_cpp

create_chat_completion 

create_chat_completion(
    messages: Json,
    *,
    model_path: Optional[String] = None,
    repo_id: Optional[String] = None,
    repo_filename: Optional[String] = None,
    args: Optional[Json] = None
) -> Json

Generate a chat completion from a list of messages.

The model can be specified either as a local path, or as a repo_id and repo_filename that reference a pretrained model on the Hugging Face model hub. Exactly one of model_path or repo_id must be provided; if model_path is provided, then an optional repo_filename can also be specified.

For additional details, see the llama_cpp create_chat_completions documentation.

Parameters:

  • messages (Json) –

    A list of messages to generate a response for.

  • model_path (Optional[String], default: None ) –

    Path to the model (if using a local model).

  • repo_id (Optional[String], default: None ) –

    The Hugging Face model repo id (if using a pretrained model).

  • repo_filename (Optional[String], default: None ) –

    A filename or glob pattern to match the model file in the repo (optional, if using a pretrained model).

  • args (Optional[Json], default: None ) –

    Additional arguments to pass to the create_chat_completions call, such as max_tokens, temperature, top_p, and top_k. For details, see the llama_cpp create_chat_completions documentation.