Computer Vision

pixeltable.functions.vision

Pixeltable UDFs for Computer Vision.

Example:

import pixeltable as pxt
from pixeltable.functions import vision as pxtv

t = pxt.get_table(...)
t.select(pxtv.draw_bounding_boxes(t.img, boxes=t.boxes, label=t.labels)).collect()

mean_ap

mean_ap()

Calculates the mean average precision (mAP) over eval_detections() results.

Parameters:

• eval_dicts (list[dict]): List of dictionaries as returned by eval_detections().

Returns:

• A dict[int, float] mapping each label class to an average precision (AP) value for that class.

draw_bounding_boxes

draw_bounding_boxes(
    img: ImageT,
    boxes: JsonT,
    labels: Optional[JsonT] = None,
    color: Optional[str] = None,
    box_colors: Optional[JsonT] = None,
    fill: bool = False,
    width: int = 1,
    font: Optional[str] = None,
    font_size: Optional[int] = None,
) -> ImageT

Draws bounding boxes on the given image.

Labels can be any type that supports str() and is hashable (e.g., strings, ints, etc.).

Colors can be specified as common HTML color names (e.g., 'red') supported by PIL's ImageColor module or as RGB hex codes (e.g., '#FF0000').

If no colors are specified, this function randomly assigns each label a specific color based on a hash of the label.

Parameters:

• img (ImageT) –

  The image on which to draw the bounding boxes.

• boxes (JsonT) –

  List of bounding boxes, each represented as [xmin, ymin, xmax, ymax].

• labels (Optional[JsonT], default: None ) –

  List of labels for each bounding box.

• color (Optional[str], default: None ) –

  Single color to be used for all bounding boxes and labels.

• box_colors (Optional[JsonT], default: None ) –

  List of colors, one per bounding box.

• fill (bool, default: False ) –

  Whether to fill the bounding boxes with color.

• width (int, default: 1 ) –

  Width of the bounding box borders.

• font (Optional[str], default: None ) –

  Name of a system font or path to a TrueType font file, as required by PIL.ImageFont.truetype(). If None, uses the default provided by PIL.ImageFont.load_default().

• font_size (Optional[int], default: None ) –

  Size of the font used for labels in points. Only used in conjunction with non-None font argument.

Returns:

• ImageT –

  The image with bounding boxes drawn on it.

eval_detections

eval_detections(
    pred_bboxes: JsonT,
    pred_labels: JsonT,
    pred_scores: JsonT,
    gt_bboxes: JsonT,
    gt_labels: JsonT,
    min_iou: float = 0.5,
) -> JsonT

Evaluates the performance of a set of predicted bounding boxes against a set of ground truth bounding boxes.

Parameters:

• pred_bboxes (JsonT) –

  List of predicted bounding boxes, each represented as [xmin, ymin, xmax, ymax].

• pred_labels (JsonT) –

  List of predicted labels.

• pred_scores (JsonT) –

  List of predicted scores.

• gt_bboxes (JsonT) –

  List of ground truth bounding boxes, each represented as [xmin, ymin, xmax, ymax].

• gt_labels (JsonT) –

  List of ground truth labels.

• min_iou (float, default: 0.5 ) –

  Minimum intersection-over-union (IoU) threshold for a predicted bounding box to be considered a true positive.

Returns:

• JsonT –

  A list of dictionaries, one per label class, with the following structure:

• JsonT –

  ```python

• JsonT –

  { 'min_iou': float, # The value of min_iou used for the detections 'class': int, # The label class 'tp': list[int], # List of 1's and 0's indicating true positives for each # predicted bounding box of this class 'fp': list[int], # List of 1's and 0's indicating false positives for each # predicted bounding box of this class; fp[n] == 1 - tp[n] 'scores': list[float], # List of predicted scores for each bounding box of this class 'num_gts': int, # Number of ground truth bounding boxes of this class

• JsonT –

  }

• JsonT –

  ```