What is rotoscoping in VFX, and why does it take so long?

Rotoscoping (roto) is drawing shapes around something in a shot, frame by frame, so it can be separated from everything else. The result is a matte: a black-and-white image that tells the compositing software which pixels belong to the subject. It’s slow because the shapes have to change on almost every frame, and the hardest parts of a subject (hair, fast motion, things passing in front of it) are the parts that change the most.

This post covers what roto is, how it differs from keying and AI masks, the tools artists use, why it takes as long as it does, what a good matte should give you, and where roto sits in a VFX pipeline.

What rotoscoping means today

In VFX and post-production, rotoscoping means creating a matte for an element in live-action footage by drawing and animating shapes. You roto a person so you can put a new sky behind them, a car so you can grade it separately, a boom mic so a paint artist knows exactly what to remove.

The word is older than VFX. Max Fleischer filed a patent in 1915 for a “method of producing moving-picture cartoons”, granted on October 9, 1917: live action was filmed, projected frame by frame onto a surface, and traced by animators. The device was the rotoscope. The tracing idea survived; the target changed. Instead of tracing a performer to make a drawing, a roto artist traces a subject to make a matte.

Roto vs keying vs AI masks

All three produce a matte. They differ in where the matte comes from.

Method	Where the matte comes from	Works best on	Weak on
Keying	A color the software can remove (green or blue screen)	Shots planned for VFX, with an even, well-lit screen	Spill, uneven screens, anything not in front of the screen
Rotoscoping	Shapes an artist draws and animates	Any background; full control over every edge	Time; very fine detail like flyaway hair
AI masks	A model that segments the subject after you click, draw or describe it	Clear subjects with solid edges, quick isolation	Hair, motion blur, transparency, occlusion; edges that chatter from frame to frame

In practice these get combined. A compositor might key a green-screen shot, roto the arm that crossed outside the screen, and use an AI mask as a starting point or a garbage matte. The AI side splits into two different kinds of model, segmentation (hard masks) and matting (soft edges). We explain the difference in Is AI rotoscoping good enough for client work?.

The tools of roto: splines, shapes, feathering and motion blur

A roto shape is a closed spline: a curve defined by a small number of control points, usually Bezier or B-spline. The artist places the points on a keyframe, moves to another frame, adjusts the points to match, and the software interpolates in between. Then the artist checks the in-between frames and fixes where the interpolation drifted.

A few ideas carry most of the craft:

• Many simple shapes, not one complicated one. A person is rarely one shape. An artist breaks them into a head, torso, upper arm, forearm, hand, each shape moving more simply than the whole body. Simple shapes need fewer keyframes and hold up better.
• Feathering. Real edges aren’t razor sharp. Feather softens the edge of a shape, and good roto tools let you set it per point, so the edge of a sleeve can be soft while the edge of a belt buckle stays hard.
• Motion blur. When a subject moves fast, its edge on the plate is a smear, not a line. Roto tools can blur the shape along its motion so the matte matches the blur in the photography. Getting it wrong shows up as a hard outline around a blurred hand.
• Tracking. Artists rarely animate every point by hand. They track the shot (points, or planes with a planar tracker) and use that motion to carry the shapes, then correct.

An fxguide article on Foundry’s SmartRoto summed up why artists care so much about shapes: “splines remain the currency of roto, and they are subpixel-accurate.” A spline can be edited, handed to another artist, and re-rendered at any resolution. A pixel matte can only be painted.

Why rotoscoping takes so long

Roto is slow because accuracy has to hold on every frame, and a subject changes on every frame. The most-cited measurement comes from the Roto++ paper (SIGGRAPH 2016, a collaboration between The Foundry, University College London and the University of Bath): “An experienced artist can rotoscope on average 15 frames per day, depending on the complexity of the scene.”

Treat that as an average across professional feature work, not a rate you can budget from. A simple shot moves much faster. A difficult one can be slower. For scale: at 24 fps, 15 frames is well under a second of footage.

The same paper describes roto as representing “a significant investment in person-hours”, and notes that the work “is often outsourced to dedicated artists, a lengthy process that creates a bottleneck and dependency in post-production pipeline.”

What makes a shot hard to roto

Four things account for most of the difficulty:

1. Hair and fine detail. Individual strands are thinner than a pixel at the edge, and semi-transparent. A spline can’t trace every strand, so artists combine a core shape with softer edge work, or pull a key for the hair where possible.
2. Motion blur. A fast hand is a smear several pixels wide. The matte has to reproduce that gradient, frame by frame, in the right direction.
3. Occlusion. When something passes in front of the subject (a hand crossing a face, a car behind a pole), the shapes have to split, hide and reappear without popping.
4. Transparency. Glass, smoke, veils, water. The subject is partly the background. There’s no single edge to draw.

Camera movement, long shots, low contrast between subject and background, and heavy grain all add to it. A good roto bid looks at these before it quotes.

What a good matte delivers

A matte is only useful if it drops into the comp without surprises. Whoever makes it, a good one should be:

• Frame-accurate. Same frame count and frame rate as the plate, starting on the same frame, ideally with the same timecode. A matte that’s one frame off looks like a bad edge on every frame.
• At plate resolution. The matte should match the plate’s pixel dimensions. Scaling a lower-resolution matte up softens exactly the edges you cared about.
• Soft where the image is soft. Graded alpha on hair and motion blur, hard where the edge is hard. White is fully the subject, black is fully not, gray is partial.
• Temporally stable. Edges that don’t chatter or boil from frame to frame. Per-frame accuracy isn’t enough; people see flicker before they see a slightly wrong edge.
• Clearly labeled. If the matte is delivered as the alpha of an RGBA file, say whether it’s straight (color independent of alpha) or premultiplied (color already multiplied by alpha). Mixing them up is the classic cause of dark or bright fringes. We go deeper in Alpha channels, ProRes 4444 and EXR.
• Editable, when the job needs it. On many projects the roto shapes are part of the deliverable, so a compositor can adjust the edge later without redoing the work.

Where roto sits in the VFX pipeline

Roto usually happens early, right after the plates are turned over, because other departments depend on it. Paint and prep artists use roto to define what gets removed (a rig, a wire, a tracking marker) and what must be protected. Compositors use it to layer the subject over new backgrounds, grade parts of the frame separately and hold effects behind the talent. Colorists use roto-style windows to isolate faces and objects in the grade.

On larger productions, roto and paint are often a separate department or an outside vendor, as the Roto++ paper notes. On commercials, music videos and indie work, the editor, motion designer or compositor usually does it themselves, which is why the tools built into After Effects, Premiere Pro and DaVinci Resolve matter so much. We compare those in Object Matte vs Magic Mask vs Object Mask. Roto is also the first step of many cleanup jobs, covered in How to make a clean plate.

How AI is changing rotoscoping

AI tools now propose a mask from a click, a box or a text description, and propagate it through the shot. Segmentation models such as Meta’s SAM 3 can “detect, segment, and track objects using text or visual prompts”, and After Effects, Premiere Pro and DaVinci Resolve Studio all ship an AI masking tool of their own.

What this changes: the first pass on a simple subject can be far quicker than drawing it by hand, and roto becomes practical on shots where nobody would have budgeted for it. What it doesn’t change: hair, motion blur, occlusion and transparency are still the hard cases, most AI tools output pixel mattes rather than editable splines, and a person still has to judge whether the edge holds up. The honest assessment is in Is AI rotoscoping good enough for client work?.

The change also raises a real question about the entry-level work many roto artists start with. We think the time it frees should go to the people doing the work, and we’ve written about that in Will AI replace roto artists?

In nolanlabs, roto sits inside every job: you point at the subject, and it returns a matte that follows it through the shot, which you keep refining in your own tools. More under what you get back.

If you’re learning roto, the practical takeaway is the same with or without AI: study the plate before you draw, break the subject into simple shapes, match the motion blur, and check the matte over a contrasting background at full resolution, frame by frame, before you hand it on.

Questions

What is the difference between rotoscoping and keying?

Keying pulls a matte from a color, usually a green or blue screen, so the background does most of the work. Rotoscoping draws the matte by hand with shapes, so it works on any background but takes far longer. Many shots use both: a key for the easy areas and roto for the parts the key can't hold.

How long does rotoscoping take?

It depends heavily on the shot. The Roto++ paper (SIGGRAPH 2016) reports that an experienced artist rotoscopes about 15 frames per day on average, depending on the complexity of the scene. A locked-off shot of a solid object goes much faster than a handheld shot of someone with loose hair moving quickly.

What file does a rotoscoping job deliver?

Usually a matte: a grayscale image sequence or an alpha channel where white is the subject, black is everything else and gray is partial coverage. It is typically delivered as an EXR or PNG sequence or a ProRes 4444 file with alpha, at the plate's resolution and frame count, and sometimes with the roto shapes themselves for further editing.

Sources

1. Roto++: Accelerating Professional Rotoscoping using Shape Manifolds, by Li, Viola, Starck, Brostow and Campbell (SIGGRAPH 2016) · accessed 2026-09-24
2. US1242674A, Method of producing moving-picture cartoons, by Max Fleischer (Google Patents) · accessed 2026-09-24
3. Foundry's SmartRoto: AI-assisted roto that aims to work the way you do, fxguide (July 16, 2026) · accessed 2026-09-24
4. SAM 3 repository, Meta (facebookresearch/sam3, GitHub) · accessed 2026-09-24

Written by the nolanlabs team. nolanlabs does AI shot work for post-production; product names mentioned belong to their owners. Tool details were accurate on the date shown above. Check the vendor’s documentation before relying on them.