From Screenplay to 3D Motion: Building PF Stage’s Multi-Actor Animation Pipeline

I wanted characters in a virtual set to act out screenplay directions without putting on a mocap suit—or animating every frame by hand. That became the starting point for a new experiment inside PF Stage, the virtual production tool I’m building as a solo developer.

The goal is simple: turn a written scene into structured, editable motion tracks. Getting there takes more than asking an AI to “make the character move.” Timing, position, rotation and interactions have to work together.

1. Read the screenplay as structured action

An LLM analyzes the scene directions and extracts actor actions, timestamps, root positions in XYZ space and interaction tags such as grip, support, impact and contact. These become explicit data that the next stage can work with—not just another paragraph of instructions.

2. Build an editable multi-actor timeline

The pipeline lays out discrete joint rotations and root trajectories across a 30-second, multi-track scene. The editor lets me inspect actor poses, positions and interactions along that timeline, rather than treating the output as a finished clip that cannot be changed.

3. Bake the motion at 60fps

A dense motion engine turns those keyframes into frame-by-frame motion data. It uses PCHIP cubic Hermite interpolation for root trajectories, with SQUAD/SLERP quaternion interpolation for joint rotations.

The accompanying video shows the screenplay input, multi-actor timeline and a separate Taekkyeon motion playback test. “60fps” describes the baked motion data; this showcase is a 30fps screen recording.

What works—and what I’m refining

The core text-to-structured-motion workflow is running across my scene tests. It remains a proof of concept, not a claim that every generated movement is production-ready.

Two areas need more work: foot sliding, which needs stronger IK ground locking, and extreme joint rotations, which need better limits. Those are the next pieces I’m refining as I prepare for a public release.

What would you test first?

If you work with animation, previs or virtual production, I’d love your feedback. Which would matter most in your workflow: contact timing, foot stability or control over the generated motion curves?

PF Stage is in active development. Share a scene direction you would want to try.