AI-powered video segmentation using Meta's SAM2 for automated rotoscoping in After Effects. BodyFX automatically detects and tracks subjects frame-by-frame, exporting segmentation data as animatable shape layers via 7 ExtendScript (.jsx) files.
BodyFX bridges the gap between recent AI research and practical video production needs. By leveraging Meta's Segment Anything Model 2 (SAM2), it automates one of the most time-consuming tasks in visual effects: rotoscoping and subject isolation.
Manual rotoscoping in video production is extremely time-intensive, often requiring hours of frame-by-frame work. BodyFX automates this process using SAM2-based AI segmentation, significantly reducing the time required while maintaining professional quality.
Leverage Meta's SAM2 model for automatic subject detection and tracking without manual rotoscoping setup
Seamless workflow integration with Adobe After Effects for professional VFX work
Early-stage exploration of real-time background removal for live streaming — currently a stub with basic error handling
Built for production with scalability and speed in mind
BodyFX implements a sophisticated pipeline that handles everything from video ingestion to After Effects integration, all optimized for production workflows.
End-to-end workflow from video input to animated shape layers in After Effects
# Basic usage
$ bodyfx-process video.mp4 models/sam_vit_h.pth
# Multi-subject with GPU acceleration
$ bodyfx-process video.mp4 models/sam_vit_h.pth \
--multi-subject --device cuda --batch-size 4
The custom ExtendScript (JSX) integration provides a native After Effects experience:
An OBS plugin for real-time background removal is planned but currently in early exploration. Only a basic C++ error handler file exists as scaffolding. The primary focus of BodyFX is the After Effects segmentation pipeline described above.
Note: The OBS plugin is not yet functional. This section describes planned future work.
BodyFX applies recent computer vision research to practical production workflows, making GPU-accelerated AI segmentation accessible to creative professionals.
Transformer-based segmentation model for precise subject isolation
Smart tracking maintains segmentation quality across frames
Point or bounding box prompts for targeted segmentation
Automatic memory management for long videos
Quick key frame previews before full processing
pip installable with automatic dependency management
The project implements several optimization strategies to handle production workloads:
BodyFX has been designed with production pipelines in mind, offering practical solutions for common VFX challenges:
Quick rough cuts for editorial, clean plates for VFX
Professional streams without green screens, virtual backgrounds
YouTube creators, social media effects, dynamic backgrounds
Professional video calls, virtual presentations, online teaching
Exploring integration with other AI models for enhanced capabilities like 3D depth estimation, motion prediction, and style-aware segmentation.
Use the live surface or the source as the next level of proof. The goal here is not to end on a marketing flourish, but to make the next inspection step obvious.