Skip to content

gemini

pixeltable.functions.gemini

Pixeltable UDFs that wrap various endpoints from the Google Gemini API. In order to use them, you must first pip install google-genai and configure your Gemini credentials, as described in the Working with Gemini tutorial.

generate_content async 

generate_content(
    contents: String,
    *,
    model: String,
    config: Json | None = None,
    tools: Json | None = None
) -> Json

Generate content from the specified model. For additional details, see: https://ai.google.dev/gemini-api/docs/text-generation

Request throttling: Applies the rate limit set in the config (section gemini.rate_limits; use the model id as the key). If no rate limit is configured, uses a default of 600 RPM.

Requirements:

  • pip install google-genai

Parameters:

  • contents (String) –

    The input content to generate from.

  • model (String) –

    The name of the model to use.

  • config (Json | None, default: None ) –

    Configuration for generation, corresponding to keyword arguments of genai.types.GenerateContentConfig. For details on the parameters, see: https://googleapis.github.io/python-genai/genai.html#module-genai.types

  • tools (Json | None, default: None ) –

    An optional list of Pixeltable tools to use. It is also possible to specify tools manually via the config['tools'] parameter, but at most one of config['tools'] or tools may be used.

Returns:

  • Json –

    A dictionary containing the response and other metadata.

Examples:

Add a computed column that applies the model gemini-2.0-flash to an existing Pixeltable column tbl.prompt of the table tbl:

>>> tbl.add_computed_column(response=generate_content(tbl.prompt, model='gemini-2.0-flash'))

generate_images async 

generate_images(
    prompt: String, *, model: String, config: Json | None = None
) -> Image

Generates images based on a text description and configuration. For additional details, see: https://ai.google.dev/gemini-api/docs/image-generation

Request throttling: Applies the rate limit set in the config (section imagen.rate_limits; use the model id as the key). If no rate limit is configured, uses a default of 600 RPM.

Requirements:

  • pip install google-genai

Parameters:

  • prompt (String) –

    A text description of the images to generate.

  • model (String) –

    The model to use.

  • config (Json | None, default: None ) –

    Configuration for generation, corresponding to keyword arguments of genai.types.GenerateImagesConfig. For details on the parameters, see: https://googleapis.github.io/python-genai/genai.html#module-genai.types

Returns:

  • Image –

    The generated image.

Examples:

Add a computed column that applies the model imagen-3.0-generate-002 to an existing Pixeltable column tbl.prompt of the table tbl:

>>> tbl.add_computed_column(response=generate_images(tbl.prompt, model='imagen-3.0-generate-002'))

generate_videos async 

generate_videos(
    prompt: String | None = None,
    image: Image | None = None,
    *,
    model: String,
    config: Json | None = None
) -> Video

Generates videos based on a text description and configuration. For additional details, see: https://ai.google.dev/gemini-api/docs/video-generation

Request throttling: Applies the rate limit set in the config (section veo.rate_limits; use the model id as the key). If no rate limit is configured, uses a default of 600 RPM.

Requirements:

  • pip install google-genai

Parameters:

  • prompt (String | None, default: None ) –

    A text description of the videos to generate.

  • image (Image | None, default: None ) –

    An optional image to use as the first frame of the video. At least one of prompt or image must be provided. (It is ok to specify both.)

  • model (String) –

    The model to use.

  • config (Json | None, default: None ) –

    Configuration for generation, corresponding to keyword arguments of genai.types.GenerateVideosConfig. For details on the parameters, see: https://googleapis.github.io/python-genai/genai.html#module-genai.types

Returns:

  • Video –

    The generated video.

Examples:

Add a computed column that applies the model veo-2.0-generate-001 to an existing Pixeltable column tbl.prompt of the table tbl:

>>> tbl.add_computed_column(response=generate_videos(tbl.prompt, model='veo-2.0-generate-001'))

invoke_tools 

invoke_tools(tools: Tools, response: Expr) -> InlineDict

Converts an OpenAI response dict to Pixeltable tool invocation format and calls tools._invoke().