University of Göttingen's SiLVi Turns Animal Videos Into Relational Training Data

Researchers training models to recognize social behavior need labels that connect an action to its participants, location, and duration. Conventional behavior-logging tools record events such as grooming without locating the animals in the frame, while many computer-vision annotators track individuals without describing their relationships. Neither output captures a complete interaction such as one primate grooming another or looking toward a food box.

A team from the University of Göttingen and the German Primate Center has released SiLVi, an open-source desktop application that combines behavioral and spatial annotation. According to the accompanying paper, the tool produces dynamic scene graphs: records of entities and their relationships as they change across video frames. That structure can support training data for animal-behavior models and other systems that analyze interactions over time.

Behavior labels lose the actors

Behavior-logging tools such as BORIS can mark when an interaction occurs, but they do not tie the event to a region of the video. Models that learn spatially grounded behavior need both the event and the relevant people, animals, or objects. Other annotation tools support spatial and temporal action labels, yet many represent only individual actions. Systems based on pose keypoints can encode interactions, although reliable keypoint tracking remains difficult in cluttered or changing environments.