Alright, that’s a long title, but that’s the shortest I could come up with to describe this tutorial.
It’s not rare that someone, somewhere, provides us with an element (logo, UI, motion graphics, precomp, anything really) with a nice alpha, but with a white or colored background.  Of course it’s not ideal, but that’s nothing we can’t get around if we know how.
Then, by using the same logic, I’ll try to explain how to invert almost any other comp operation.

Additionally, although it’s demonstrated here for Nuke, this would work in most compositing packages.

Transparent Element is not on a Black Background.

I just came across a little Gizmo on Nukepedia called ColorPremult. I was surprised to realize it didn’t seem to be there earlier, seing how often I have been given elements on a colored background. While the Gizmo seem to work as advertised, I thought we could go a step farther and make it work on ANY background, not only with a solid color.

Breaking it down.

Let’s assume we have been given this beautiful TV news banner to drop onto some footage of a TV show, along with its alpha:

TVnews TVnews_alpha

If we try to just premult the two together, we will obtain the following result, which is wrong (you can still see the background in it):

TVnews_premult

What we should have been given was the banner onto a black background along with its alpha. This, instead is that element already comped over a background. To obtain our element, we basically need to “un-comp” it from the background.

2016-06-26_17h00_46

The operation happening here is using 3 elements to generate a 4th: The Background (1), Foreground (2) and ForegroundAlpha (3) are assembled into the result (4)

Merge nodes are just simple math, and luckily Nuke tells us the math of every single merge mode in the node. By looking at that, it tells us that an Over is doing the following: A+B*(1-a)
Which can also be read as: Result = Foreground + Background * (1 – AlphaOfA)

What that means is that we will be able to un-comp this at the condition that we have the background. In the case of any solid color, it’s easy, just need to pick the color. In the case of something else, it’s about luck, but I had a few agencies giving me elements comped on top of a frame I had given them previously, in which case I did have the background. Without the background there is nothing more this tutorial can do for you.

Just like in middle school, to obtain our proper Foreground, we’ll need to solve a little equation:
Result = Foreground + Background * (1 – AlphaOfA)
Result – Background * (1 – AlphaOfA) = Foreground
(Hey, Middle school was a while ago, if you’re not sure what I did there review it over here: http://www.mathsisfun.com/algebra/equations-solving.html )

Now let’s apply it in Nuke:
(Turns out, we have a merge that does Background * (1 – AlphaOfA): Stencil)
2016-06-26_17h32_18We now have a correct element:
TVnews_correct

The same operation in flame. Thanks to Ivar for the screenshot.

The same operation in flame. Thanks to Ivar for the screenshot.

[code collapse=”true”] set cut_paste_input [stack 0] version 9.0 v5
Group {
inputs 0
name Group1
selected true
xpos 114
ypos -52
postage_stamp true
}
Expression {
inputs 0
expr0 1
expr1 x%20>10?1:0
expr2 x%20>10?1:0
name Expression1
selected true
xpos -250
ypos -42
}
Transform {
translate {-260 -232}
rotate -15
center {960 540}
name Transform1
selected true
xpos -250
ypos -16
}
Ramp {
p0 {1564 192}
p1 {880 184}
color {1 0.88 1 1}
color_panelDropped true
name Ramp1
selected true
xpos -250
ypos 10
}
Roto {
output alpha
replace true
curves {{{v x3f99999a}
{f 0}
{n
{layer Root
{f 2097152}
{t x44700000 x44070000}
{a pt1x 0 pt1y 0 pt2x 0 pt2y 0 pt3x 0 pt3y 0 pt4x 0 pt4y 0 ptex00 0 ptex01 0 ptex02 0 ptex03 0 ptex10 0 ptex11 0 ptex12 0 ptex13 0 ptex20 0 ptex21 0 ptex22 0 ptex23 0 ptex30 0 ptex31 0 ptex32 0 ptex33 0 ptof1x 0 ptof1y 0 ptof2x 0 ptof2y 0 ptof3x 0 ptof3y 0 ptof4x 0 ptof4y 0 pterr 0 ptrefset 0 ptmot x40800000 ptref 0}
{curvegroup RectangleCusped1 512 bezier
{{cc
{f 8192}
{px 1
{0 0}
{xc28c0000 x43940000}
{0 0}
{0 0}
{x44368540 x4393fffe}
{0 0}
{0 0}
{x44368540 x4319fffe}
{0 0}
{0 0}
{xc28c0000 x431a0000}
{0 0}}}
{cc
{f 8192}
{px 1
{0 0}
{0 0}
{0 0}
{0 0}
{x445de9d0 x400b2f80}
{0 0}
{0 0}
{x445e4bec xc0569700}
{0 0}
{0 0}
{0 0}
{0 0}}}}
{tx 1 x441e8000 x43610000}
{a osw x41200000 osf 0 str 1 spx x44700000 spy x44070000 sb 1 tt x41100000}}}}}}
toolbox {selectAll {
{ selectAll str 1 ssx 1 ssy 1 sf 1 }
{ createBezier str 1 ssx 1 ssy 1 sf 1 sb 1 tt 4 }
{ createBezierCusped str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ createBSpline str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ createEllipse str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ createRectangle str 1 ssx 1 ssy 1 sf 1 sb 1 tt 8 }
{ createRectangleCusped str 1 ssx 1 ssy 1 sf 1 sb 1 tt 9 }
{ brush str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ eraser src 2 str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ clone src 1 str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ reveal src 3 str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ dodge src 1 str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ burn src 1 str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ blur src 1 str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ sharpen src 1 str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ smear src 1 str 1 ssx 1 ssy 1 sf 1 sb 1 }
} }
toolbar_brush_hardness 0.200000003
toolbar_source_transform_scale {1 1}
toolbar_source_transform_center {960 540}
colorOverlay {0 0 0 0}
lifetime_type "all frames"
motionblur_shutter_offset_type centred
source_black_outside true
name Roto1
selected true
xpos -250
ypos 46
}
Premult {
name Premult1
selected true
xpos -250
ypos 84
}
Text2 {
font_size_toolbar 100
font_width_toolbar 100
font_height_toolbar 100
message "Special News"
old_message {{83 112 101 99 105 97 108 32 78 101 119 115}
}
box {309 169.5 903 270.5}
transforms {{0 2}
}
cursor_position 8
scale {1 1}
cursor_initialised true
initial_cursor_position {{309 270.5}
}
group_animations {{0} imported: 0 selected: items: "root transform/"}
animation_layers {{1 11 960 540 0 0 1 1 0 0 0 0}
}
color {0 0 0 1}
color_panelDropped true
enable_shadows true
shadow_opacity 0.835
name Text1
selected true
xpos -250
ypos 110
}
Text2 {
font_size_toolbar 200
font_width_toolbar 100
font_height_toolbar 130
tracking_toolbar -0.168
opacity 0.335
message TV
old_message {{84 86}
}
box {19.6953125 99.5 250.3046875 360.5}
transforms {{0 2}
}
font_size_values {{0 200 1 200 0 101 1 101}
}
font_height_values {{0 130 1 130 0 102 1 102}
}
kern_values {{0 0 1 0 0 0.002 1 0.002}
}
tracking_values {{1 -0.168}
}
font {{ Adobe Gothic Std : B : AdobeGothicStd-Bold.otf : 0 }}
font_size 200
font_height 130
tracking -0.168
scale {1 1}
cursor_initialised true
initial_cursor_position {{19.6953125 360.5}
}
group_animations {{0} imported: 0 selected: items: "root transform/"}
animation_layers {{1 11 960 540 0 0 1 1 0 0 0 0}
}
color {1 0 0 1}
color_panelDropped true
enable_shadows true
name Text2
selected true
xpos -250
ypos 136
}
CheckerBoard2 {
inputs 0
format "1920 1080 0 0 1920 1080 1 HD_1080"
name CheckerBoard1
xpos 60
ypos -8
}
Shuffle {
alpha black
name Shuffle1
xpos 60
ypos 64
}
Merge2 {
inputs 2
name Merge1
xpos 60
ypos 136
}
Output {
name Output1
xpos 60
ypos 236
}
end_group
set Nc0008000 [stack 0] push $Nc0008000
CheckerBoard2 {
inputs 0
format "1920 1080 0 0 1920 1080 1 HD_1080"
name BG
selected true
xpos 272
ypos -52
}
Merge2 {
inputs 2
operation stencil
name Merge1
selected true
xpos 193
ypos 65
postage_stamp true
}
Merge2 {
inputs 2
operation minus
name Merge2
selected true
xpos 114
ypos 177
postage_stamp true
}[/code]

Using it on other effects

By following the same principle, you can undo almost any comp operation. I have used it to remove a semi-transparent logo from a whole image sequence. (same math as above almost, except that time the Background was the unknown)
Another time I used it to recover the footage in a dissolve. Some video footage was dissolving to a still frame of the commercial’s brand logo. I had to replace that logo frame with the Chinese version of the Logo for versioning, but the production company was unable to provide us with the footage without the dissolve.
Since a dissolve is using the same math as an over, but with an animated solid alpha (from 0% to 100% white), I was able to isolate the background footage and replace the logo seamlessly.

Limitations

As mentioned previously, in order to reverse a comp operation, you need to possess every other elements involved in the operation.
If some data is entirely missing (let’s say a part of the background behind a 100% opaque element) you won’t be able to recover it (99.9% would be fine, in 32bit).
Finally, bit depth, clamping and compression might limit how much of an operation is undoable.

Bonus for Flame Users

The famous Ivar Beer from the Logik Flame Users group on Facebook has built the demonstrated operation into a Matchbox Shader: https://logik-matchbook.org/shader/crok_uncomp