Rendering Layer Masks (for Photoshop) In Rhino

Using Layer Masks to manage visibility is an essential Photoshop technique, but cutting out the masks is painful and time consuming. Luckily we can easily prepare them in Rhino.

In this tutorial we’ll see how to fake some realistic effects in Photoshop:

In order to do this, we will depend heavily on very accurate Layer Masks.
As you’ve seen so far, cutting out masks in Photoshop is no fun and a very time-consuming process. That is why being able to render layer masks directly from Rhino is an essential technique to produce good photo-montages.

Luckily it’s easy to do. Let’s first try to understand what we want to do here.

The Main Idea

Layer masks are basically black/white (grayscale) images that define what’s seen (white), what’s invisible (black) and what’s transparent (greys). As such, we can easily set our Rhino scene to render such black and white images.

For example, if we want to have a mask that cuts out all windows, we’ll render an image similar to the one on the left, and then use it in Photoshop as a layer mask:

ps-rhino-concept

Now, let’s see how we do that, and how we apply this concept to achieve different effects.

Download Files

If you’ve finished the previous tutorial, then you can just continue by using that Rhino scene and the rendering you’ve made. If you didn’t do it -and I strongly recommend you do it first- you can download the following files to get started:

Download Tutorial Files (1.7Mb .zip)
(important: better use the files from the previous tutorial if you have them!)

Preparing Rhino File To Render Masks

At first, know that in order to render masks, we will have to change materials and some render settings in our Rhino file. So the first thing to do is:

Make A Copy Of Your Rhino File

It’s very important to work on a separate Rhino file, so you wouldn’t use your nice materials and render settings. So before you proceed, make sure to make a copy of the Rhino file first:

Turn Off Lights, Make Background Black

Since we want pure white on black rendering, without any shadow or lighting effects, we need to turn off all our lights (skylight also) and make our background black:

Create Black And White Materials

Finally, we want to create two new materials. One completely black and other emissive white. “Emissive” means that not only we want to set its color to white, but also Emission color in material’s Advanced Settings. This will make sure that our white objects appear as pure 100% flat white:

Now we’re set to start masking!

Rhino To Photoshop Workflows

Cutting Out Our Rendering From Background

First typical thing we want to do is cut out our rendering from background.
Now you probably may ask yourself two things:

  1. Why we want to cut it out from the background at all? The reason is that in many cases we want to do something to the background image, ie. fade it away, or add more elements than what’s there in the original photo. So it’s always good to have these two layers (our rendering, and the background) separate.
  2. Doesn’t Rhino offer “transparent background” option in Render Properties? Yes it does, but it doesn’t work very well. Most notably, if you use any reflections, you will notice a bunch of unwanted results (ie. your reflections being transparent, too). Since the implementation of this option (as well as many others) is very clumsy and unprofessional in Rhino, we better to do manually.

The idea is simple. We’ll render an image where the background (including the ground object) is black, and our building is white. Note that we still want to keep the glass invisible! Here’s what we need to render, and how we’ll use it in Photoshop:

back-mask

Most of the things you already know how to do.
The only new thing really is ability to paste something into the mask directly (hint: use ALT Key + Click on a mask thumbnail to reveal the mask view in Photoshop).

Here’s how to render the mask, first:

And then how to use it in Photoshop:

Pro tip: If your renderings have jagged pixelated edges, then open Render Properties and change Anti-Aliasing settings (the higher, the better – but it will slow down the rendering):

prop-aa 

Make Our Building Cast Shadow On Background

What I mean by this, in order for our building to sit naturally in its surroundings, we need to simulate the shadow the building casts on it, too. That’s why we have the ground object in Rhino, which receives the shadow of the building. Now we’ll make this ground object look, in Photoshop, as its casting the shadow on the grass behind the building.

The workflow is basically:

For details of understanding blending modes, check this tutorial. Right now we’ll just focus on the required workflow:

Faking Glass Reflections

This is extremely useful technique, given that making good reflections in Rhino is not very straightforward, and can be very time consuming. On the other hand, if we fake them in Photoshop, we can easily experiment and adjust results in real-time.

The workflow is:

ps-rhino-concept

So let’s get started (and finished):

Adding Texture Detail

Very often we may want to add texture details in Photoshop. The reason for that is that -often- the textures we apply to materials may look too repetitive. By adding another layer of texture in Photoshop we can add irregularities that look much more natural.

For this, we’ll again render a mask in Rhino (in this case for our concrete objects) and then use it as a mask for a layer group. Then we’ll put our texture images inside the layer group and play with blending mode and opacity until it looks good.

Final Image

Finally, we may want to complement the image with a person on the left (to balance the asymmetry a bit) and use adjustment layer to color-balance entire image.
Here’s how the final image may look like:

render-final-shot

And in case you got lost along the way, you can download the final PSD file and inspect it:

Download Final Photoshop File (3.1Mb .zip)

 

 

 

 

 

 

Any questions?

Leave a Reply

You must be logged in to post a comment.