Projection Correction

Some time ago I had to take a bunch of screenshots of the TTS VR for advertising purposes.
They needed a lot of screenshot for various formats (web banners, magazines, …),
so they tend to recrop the screenshots I send them a lot, depending on the need and space,
because of that I have to add some extra space outside of the actual screenshots, which I simply do by increasing the fov of the camera I render the screenshots with.

The thing is though, that the higher the fov, the more perspective distortion you get away from the center.
(a logical result for a projection of a 3d space onto flat plane)
This isn’t essentially a bad thing though, it’s actually correct, if the image is viewed from straight in front of the center, and at the right distance, so it’s looked at with the same angular size as the image was made (the fov).

The problem, is that these screenshots sometimes get cropped in extreme ways, by which I mean, far to one side, moving focus away from the center.
This causes the image to look strange, as the content is skewed and stretched.

So I decided to make something to counter that.

I figured that if the 2d space that was projected on was a spherical shell instead of a flat plane, there would be no distortion.
A spherical shell however can not be mapped to a flat plane without distortion (obviously, just look at maps of the earth),
but, a cylindrical shell totally can.

So, using math and logic (whoo) I created a post effect that manipulates the image as if it was projected on a cylinder (with either a horizontal or a vertical axis), and then unfolded.

This way it can make sure there is no distortion either horizontally, or vertically, depending on whether the image is in landscape or portrait.
Because of this, you can move focus in this direction (horizontal or vertical), without it looking off.
A side effect of this is that you can easily stitch together screenshots made by rotating around the cylinder-axis.

rendered with projection correctionAn extreme example, the horizontal fov is more than 200 degrees, this isn’t even possible with normal rendering (only <180 degrees).

FullCorrection

There are a few setbacks however,
firstly, unlike normally (projecting on a flat plane), not all straight lines (in 3D) are straight in the image, some are curved,
secondly, a lot of space is wasted in the image (the black parts), because of the extra cylinder-mapping, which means you have to render at a higher resolution for the same detail.
and lastly (and definately least), there is no filtering done in the cylinder-mapping, so the result can be a bit jaggy (and moreso the bigger the fov is).

I also made a second mode, which is a bit simpler and less extreme,
as it only streches 1 of the axes, just so proportions look correcter. This one is actually more difficult to explain, just look at the pictures.

SimpleCorrection

I’ve made this post effect available for free on the Unity Asset Store:

https://www.assetstore.unity3d.com/#/content/9882

It requires a Unity Pro license in order to work though.

The power of blend maps (or whatever they’re called)

First off, I don’t actually know whether it’s called a blendmap, that’s just what I call it,
what I mean is a grayscale texture that is used to dynamically blend 2 images.

in it’s simplest form, a single numerical value defines the clipping value; for each pixel of the blendmap, if the luminosity is higher than the clipping value image1 is used, otherwise it’s image2.
Most often though, I don’t want a hard cut like that, so instead I use 1 value that roughly determines how much of image1 to use (_BlendAmount), and 1 value that defines the sharpness of the transition (_EdgeSharpness).
Which looks something like this:

float blend = blendColor + (_BlendAmount * 2 – 1);
blend = saturate(blend * _EdgeSharpness – (_EdgeSharpness – 1) * 0.5f);
result = lerp(color1, color2, blend);

Now this might not seem like much, in fact it’s pretty much just an alphamap.
But it’s strenght comes from the fact that it’s great for dynamic transitions from one image to another, in time and/or space, and that’s usefull for a lot of things.
And it’s quite simple and cheap, which is also a good thing.

For example, I used this to make a snow material, that would blend in snow with the diffuse texture, based on the surface normal, using the blendmap in the snow’s alpha channel.
And for fading HUD overlays (like the Frost Effect, and blood splatter when taking damage).

Usage in games

The first time I came into contact with this technique, was when I saw the snow in Uncharted 2 (on the internet, I never actually played the game).
They seem to be using blendmaps for the transition between snow and ground/rocks. (which gave me the idea to make a snow material)

And I believe a lot of games also use blendmaps in general for transition between terrain textures, which looks a lot better than the default fading.
(Which should actually be a little more complex, as it’s not just blending 2 textures, but any number)

But the weird thing is that I can’t seem to find anything about it, I mean, actual documentation (or just even how it’s called),
but maybe that’s just cause I’m terrible at finding things on the internet.

Dynamic Frost Effect

So a while ago, while I was making a christmas themed version of a level at work, I was playing the latest SSX after hours.

MetroXmas
The christmas level

There’s a game mode in SSX where you have to stay out of the shadow or you freeze to death, they visualise this by putting more and more frost on the screen.

I quite liked this effect, and thought it would be nice to include this in the christmas level at work.

Now in essence it’s just an image overlay, but the special thing about it is that when it fades in or out, it doesn’t just get less or more transparent, but the frost shrinks or grows.
I figured they must be using a blend map or something alike to achieve this,
which is how I then made it.

Here’s what it looks like:

sharp35 sharp65 smooth75 smooth100
this is the frost effect I made, not the one in SSX

Distortion

After making this, I thought it would be even better if the ice also distorts the view.
Since the frost effect was already a post effect, this wasn’t that difficult to implement.

What I did was create a normalmap from the frost texture.
This is used to determine the direction of the distortion, and the amount of distortion is relative to the opacity of the frost, together these define the sampling offset.
(It’s a screen space image distortion, so the distortion works by just sampling the source image with the offset.)

So this is what I got:

melting2 melting1 melting3

I’ve made this post effect available for free on the Unity Asset Store:
https://www.assetstore.unity3d.com/#/content/5337
It requires a Unity Pro license in order to work though.