There is a lot to unpack here, so buckle up: this is gonna get technical.
Intro
Yeah, we’re using headers.
So the Phenomenal @Pixel_One_ posted a link to our Wombattery Discord:
Inside this link was an example of a cool streaming overlay that someone had made that featured their Guild Wars 2 character performing its idle animation, but free of all the background stuff that you’d get in the game. It was almost as if the character had been placed before an in-game green screen and had the background removed…
If you follow me you might have seen the occasional motion graphics post I’ve done on Twitter, but you can also visit http://motiongraphics.levelcapped.com to see some of the work I’ve done as I’ve been trying to learn this art. Extracting an element from a video is like motion graphics course 102, so of course I was all over this idea.
First step was how to go about it…
Step 1: Remove the background
While this sounds obvious, it’s…kind of not. Using After Effects, I learned that if you take a still shot of a background — just the background — and then add in a video shot right in front of that exact same background, you can use a blend mode called “Divide” to basically remove elements which are shared between the two — namely, that background. This does work, but the conditions need to be extremely exact: the background cannot be busy in the least, should be a single color, and the colors in the background cannot be replicated in the subject in any way, lest you end up blowing holes in said subject.
Pixel and I scouted out the guild hall for places to shoot, but since we have the desert hall so everything is sandstone, cracked, and since GW2 has a very unique art style, wasn’t really suited for background removal. The result is…quite horrific.
So, it was back to the drawing board.
Step 2: Masking or rotoscoping
From a design standpoint, masking is the act of carving a space so something behind it shows through. It’s very flexible and very powerful, and I used masking to make this cool-ass video of my GW2 character which you can find at this link because WordPress won’t let me embed it here, meanies…
This is a time intensive process because you have to trace the thing you want to isolate, and then because that thing moves, you have to adjust the outline over time. It’s not super conducive to making a library of animations that you can use for your stream.
You may have heard of rotoscoping in animation. This involves painting over the item you want to isolate, and allowing After Effects to basically “guess” what it should be outlining. Then, then system could analyze the edges over time as the animation happened, though this would still require some tweaking here and there, and for complicated animations like emotes that could be a lot of minute adjustments.
I needed something easier.
Step 3: Banana in the tailpipe
In that original Reddit thread someone linked to a custom shader which could be used to isolate content in a game through good old fashioned chroma key sorcery. (NOTE: I apologize for the above link’s location; it’s a file hosting site and I don’t feel right re-hosting the file so if you want it, that’s where you’ll need to get it. Sorry).
Hold up…shaders are like the ultimate visual voodoo. I’ve seen them mentioned in my struggles with Unity, and they usually look something like this:
Plus, shaders are internal things, added to specific objects and the world at large while the game is being developed. What good would this proffered file do me? I’m not going crack open the .exe of the game and risk a permaban for hacking.
Turns out it’s not that hard, thanks to something called ReShade. ReShade does some extra voodoo above and beyond what shaders do, and somehow [waves hands dramatically] allows you to apply effects to your games. You can add a film grain, turn everything sepia, or black and white, and then question why you did that, but with this .fx shader in hand, I was ready to try ReShade to see if I could get a green screen into GW2.
Intermission
Now would be a good time for a bathroom break. Or just hold it and call it training for Avengers: Endgame. Whatever.
My initial trials with ReShade didn’t go too well because apparently GW2 is a snowflake that doesn’t play well with this particular app. It might work just fine with something like WoW or The Division 2, but I couldn’t get it to take with GW2.
Internet to the rescue!
Someone forked ReShade specifically for dealing with GW2, and it’s called GW2Hook. You download this puppy into the GW2 install directory, drop the green screen shader file into the appropriate directory according to GW2H’s instruction page, and then fire up the game.
When you start the game, you’ll know the app is running because you’ll see something like this:
There are two settings that you’ll need to set when using GW2H: shader settings has to be on high (not anything else, no matter how powerful your PC), and sampling needs to be set to native. I don’t know the exact names of those settings, but you’re a smart cookie and I know you’ll be able to find them.
GW2H is running all the time, and to get access to the settings you’ll need, you have to hit SHIFT + F2
Now, this is where things get weird. If you downloaded the shader and put it in the right location, you’ll see it in the select list of shaders that you can activate. Put a check-mark in the left column and the shader will activate. Simple! Now you need to mess with the settings. I can’t tell you specificall what settings to set the settings to, because it depends on where you are when you activate the shader.
Uh…second intermission
I forgot to mention…see how I’m standing outside Lion’s Arch with the vast and empty expanse behind me? You’ll need to do that: get a nice, wide open, unobstructed space behind you because what you’ll basically be doing is creating a green screen colored fog that you’ll creep up to your character. You’ll want a lot of room to play with so enclosed spaces are bad, and spaces with other objects in the FOV are also bad, especially if they’re shoulder-to-shoulder with your character. Get a lot of breathing space, and you’ll be fine.
Back to the grind – try these settings
The settings for the green screen are available even if you didn’t check the box, so if you didn’t check it and start messing with the settings and get frustrated that nothing is happening make sure the box is checked.
I set MaxFogFactor to 1, FogCurve to 175, BloomThreshold to 50, Power to 100, and Width to 0.
FogStart, though, is the lynchpin to this whole shebang. It determines where in the world the fog…eh…starts. A large number means “Somewhere in Orr” while the value I have — 0.01 — puts it a few feet behind the character.
That fog rolled in pretty quickly, so you might not have a lot of room to play around with values. This is why I’m standing on a ledge above the sea outside of Lion’s Arch.
I have also hidden the UI, because the UI sits on a visual plane that’s technically outside of the reach of the fog. If you want to crop your character using Premier or AE, you can leave the UI on and cut it out later. In fact, this might be ideal since without the UI, you might have a hell of a time triggering emotes if that’s what you’re after.
Finally, to record your masterpiece, you can use (SL)OBS, XSplit, FRAPS, MS Game Bar, or your favorite video cap application. The good news is that with the green screen, you can drop the result directly into (SL)OBS or XSplit and use scene filters to remove the green, leaving you with the pretty pristine animated cut-out of your favorite characters.
Kind of a conclusion…?
So that’s kind of all there is to it, I guess. Before you leave, I want to mention that in my research I did come across the question “will this get me banned from the game?” A lot of online games don’t like people inserting things where they don’t belong, so this is a valid question. Know, then, that I found this shader thing courtesy of /r/WoodenPotatoes, who is an extremely well known and respected GW2 player and community liaison. He did some videos about ReShade and GW2Hook, so I can’t imagine that A) he didn’t ask someone at ANet about it at some point, or B) someone at ANet didn’t see his video on the subject and wave him off from promoting it. Know, however, that this is completely off-warranty, and if you use ReShade or GW2Hook and your GW2 install goes wonky, you can’t go crying to ANet.
Is it spyware? It’s all open source, so you can check it out yourself if you’re programmatically inclined, but this feeds into another point that ReShade is a Known Quantity in certain parts (I’m pretty sure I heard about it Back When in relation to Skyrim, and you might already be familiar with it yourself), and you can download profiles and shaders for all kinds of games to make them look Better(tm). I’d think that by now this practice would have been run out of town if it were more malicious than it is tedious.
Honorable Mention: Nvidia Freestyle
Again in the process of researching this, I learned about Nvidia Freestyle. If you have an Nvidia card and (ugh) use GeForce Experience, you might be familiar with Ansel, their camera-free screenshot and composition feature. Freestyle is similar to ReShade, but without the ability to write or download custom shaders. However, it does include a chroma key option which doesn’t work so hot with GW2 except in the character selection screen.
To enable this, you can open your GeForce Experience panel to make sure you have access to Freestyle. I don’t know if it’s still in beta or not, but if you don’t see it, try enabling the “experimental features”.
To use Freestyle, the hotkey combo is ALT + F3, which will show the above sidebar. You can have 3 presets, and each preset can have different filters. When you add the chroma key filter, use the disclosure triangle to open the panel and play with the settings. To turn the filter off, use the OFF button at the top (don’t just close the panel, duh).
Although Freestyle is easier to use, it’s an Nvidia feature, a (ugh) GeForce Experience feature, and is only enabled for certain games, a list of which is available on the Nvidia website.
