3ds Max + VRay Standalone is not as terrifying as you think

Update 15 June 2016 –

I have a new post up that updates this information with the latest in VRay 3.0!

Update 12 November 2013 –

There’s a collection of settings in the VRay RT parameters tab (change renderer to ActiveShade mode first, which brings up the tab you need, then switch back to Production render when you’re done) that controls whether things like XRef scenes and proxies are exported.  Enabling these makes a huge difference in how useful this cross-platform rendering method can work.

I recently had a project come in that was going to need much faster turnaround times on renderings than what I could get from Maxwell, so I added V-Ray to my workflow.  I have a love / hate relationship with V-Ray.  I ‘ve always been frustrated by how unpredictable it is – render times can be super long with the wrong settings, and it seems like you always have to be anticipating the consequences to anti-aliasing and other things as you go to keep times down and quality up.

So it was with some reluctance that I made this addition.  But I probably would have added it to the mix earlier if it wasn’t for the fact that my render farm was entirely Linux based, and it seemed like a particularly cruel form of torture to try and get VRay standalone to work with 3ds Max and render on Linux machines.  But it wasn’t too bad, and since I had trouble finding good information in various forums on how it could be done, I thought I’d post some of my experiences here for others.

Now, it looks like this might all be out of date in a couple months when V-Ray 3.0 comes out – it is supposed to have a unified core that allows render clients to be on any platform.  But it’s not here yet, so let’s do this.

If you have questions, ask in the comments and I’ll try and elaborate.

Why on Earth Would Someone Want to Do This?

There are a bunch of reasons I decided to stick with Linux on my nodes rather than switch them to Windows and use the standard 3ds Max + Vray workflow that has kept the world happy for years.  For render nodes, Linux is by far my preference for a few reasons:

• Maxwell (my primary render engine) runs about 10% faster on Linux than Windows, on the same machine.  That’s a significant boost.
• Linux can be run without a GUI, which saves lots of memory.  My base Windows 7 install uses about 2gb on boot, whereas my Ubuntu server takes about 300mb.  That’s 1.7gb more ram available for rendering.
• Once you get the hang of it, Linux is much easier to maintain on a render farm.  Each node boots from a shared install on the server which means that to make updates to the software installed, I just remote in to one node, make my updates (which updates the common install stored on the server) and reboot all the nodes.  None of the bullshit of ghosting images across machines or individually updating each node by hand.
• It costs less per node than Windows.  Most Linux varieties are free, right now Windows 7 Home OEM is going for about $100.  Plus, with the remote booting mentioned above, I don’t need a hard disk, which means about $50 saved + the ongoing electric and cooling costs.  This is offset by the need for standalone licenses of VRay which work out to about $120 if bought in a 10 pack.  And with VRay 3.0 new licensing, it looks like that offset will go away since there won’t be free nodes anymore.
• Dual booting is a pain, especially with headless render nodes.  I would much rather just keep it all simple and use all Linux or all Windows.
• The .vrscene files that VRay uses to render are plain text.  That means if you are so inclined you could do all kinds of cool stuff with Maxscript / Python, etc.

How Does it Work?

It’s actually really easy.  The basic idea is that you export a .vrscene file that represents all the geometry, cameras, shaders, etc. and is what VRay standalone uses to render.  Remember, VRay standalone knows nothing about 3ds Max and can’t use .max files at all.

But first you have to install VRay standalone.

1. Download the standalone renderer through the normal VRay download portal.  You should have access to the Maya installation after you buy licenses, which is what you want.
2. Follow the instructions included for installing standalone on the Linux and Windows machines you want it on.
3. The main ‘vray’ binary, at least for my default installation, are in the ‘/usr/autodesk’ directory.  I’m not sure why there is a ‘vrayslave’ binary in both the ‘/usr/autodesk’ path and the ‘/usr/ChaosGroup’ path.  But all the binaries I use are in the ‘usr/autodesk’ path.

Now back to 3ds Max.  There are a few ways to generate the .vrscene file:

• Use VRayRT (as activeshade or primary renderer) and use the button in the render dialog to export a .vrscene file.
• Right click in a viewport and select ‘.vrscene exporter’.  I found this to be incredibly slow and somewhat buggy at that, so I don’t recommend it.
• Use Maxscript command ‘VRayExportRTScene’ to export either by hand or by including it in a script you write.  More info on that command is here: http://help.chaosgroup.com/vray/help/rt100/render_maxscript.htm.  This is by far my top recommendation since it can be scripted to require the fewest clicks to export.

Now you have a .vrscene saved and need to render it.

• If you use Deadline for your render farm management, you are in great shape, just submit a VRay standalone job and make some tea while it renders.  If you want, customize the plugin to add some nice features – more on that in a second.
• I really really recommend you use Deadline if you are rendering cross platform.  And If you aren’t, then why are you reading this?  Just use 3ds max on Windows and save yourself the perscription headache pills.
• I haven’t tried other render managers, but Deadline will automatically swap out the paths of all your assets inside a .vrscene.  This is critical to getting things rendering cross platform, and all you have to do is set up path mapping in Deadline and go to the VRay plugin options and make sure ‘Enable Path Mapping For vrscene Files’ is set to ‘true’.
• If you know that your paths are ok, you can start it manually from the command line.  It will give you some info on the console as it goes.  An example:

  vray -sceneFile="abc.vrscene" -display=1 -autoClose=1 -displaySRGB=1 -distributed=1 -renderhost="node2;node3" -imgFile="abc.vrimg"

  • vray : name of command, you may need to specify the full path if you aren’t in the right directory already.
  • -sceneFile=”abc.vrscene” : specifies the path to the .vrscene file you created earlier.  In this case ‘abc.vrscene’
  • -display=1 : whether or not to display the rendering as it goes.  1 = yes, 0 = no.  Not that for a headless linux node, you’ll need something like x11vnc + xvfb to be able to see the output via VNC.
  • -autoClose=1 : whether or not to automatically close VRay when it finishes.  1 = yes, 0 = no and you will have to manually close the window or kill the process after finishing.  This one is important to keep on.  Otherwise VRay will just wait around forever for you to manually close it.  Trust me, it’s much cheaper to turn this on than to hire the neighbor’s kid to wait around all night and click close after every job.
  • -displaySRGB=1 : if you are using the display=1 option, this will automatically set the sRGB button.  1 = on, 0 = off
  • -distributed=1 : whether or not to use distributed rendering.  More on how this works below.
  • -renderhost=”node2;node3″ : if you are using distributed rendering, this specifies which DR helpers to use.
  • -imgFile=”abc.vrimg” : output file to write.  It will automatically detect .vrimg vs .jpg, .exr, etc.

 What are the Limitations?

Nothing is perfect, and combining VRay standalone with 3ds Max is no exception.  There are a  few glitches and features that don’t quite work.  I believe all the missing features are omissions by the exporter from Max, rather than missing functionality in the standalone renderer itself, so in theory they can be overcome with a better exporter which you could write in Maxscript or as your own plugin.  I’ll include a couple hacks later that can alleviate some of the most inconvenient things.

• Directory names for anything VRay standalone uses (input .vrscene, output image, bitmap assets, etc.) can’t have any spaces.  You need to use underscores or something else instead.
• Orthographic cameras don’t get exported right, they will render as regular perspective cameras.  You need to move the camera way out and use a super narrow FOV to get a similar effect.
• The ‘self-illumination’ parameter of the VRay material doesn’t seem to get exported.
• Proxies aren’t exported.  This is potentially the biggest problem in my mind.  Hopefully I can make it to 3.0 before needing this too much. – EDIT 12 November 2013 – this is actually possible and very easy, there’s a checkbox in the VRay RT parameters that controls whether proxies are exported.  See above for more.
• VRayHDRI maps aren’t exported.  Any HDRI lighting you use with this will be left out or substituted with a solid color.
• VRayDirt doesn’t get exported, but I hacked together a fix to use it in an ExtraTex render element.
• MultiMatte doesn’t get exported, but I hacked together a fix.
• Distributed rendering can’t mix and match Windows and Linux machines.  All clients need to be on the same OS (except Mac which should be compatible with Linux)  More on DR below.
• VRay toon doesn’t get exported.

There could definitely be others, but I didn’t run into them in my limited use so far.  Feel free to post below if you find others or if you find fixes for anything.

How Does Distributed Rendering Work?

DR is completely possible, but has a couple limitations if you are using it cross-platform.  Linux uses a different format for specifying file paths than Windows, which basically means that you can’t mix Windows and Linux in the same DR session.  I haven’t tested this with Mac yet, but I think it should work with Linux machines.

It’s a two step process for DR rendering:

1. Start the vrayslave process.  This will loiter in the background and wait for requests to help out.
2. Start VRay with the command line switches -distributed=1 -renderhost=”node2;node3″
3. Put your list of hosts in place of my example of node2;node3.  Just separate each node name (probably can use IP address too) with a semi-colon.

That’s it, it’s pretty straightforward.  If you are using Deadline, I recommend adding these switches to the Vray.py plugin so you can always be rendering with DR on the farm.

How Can I Fix Some Shortcomings of the Exporter?

As I mentioned above, the 3ds Max -> .vrscene exporter does have some omissions right now.  I found a couple Maxscript hacks that can restore what for me are the most important – the Multimatte elements and the VrayDirt in ExtraTex element.  Here’s a rundown:

If you look in the folder for the Max startup scripts, i.e. ‘C:Program FilesAutodesk3ds Max 2014scriptsStartup’ there’s a script called vrsceneexport.ms.  This is the really slow script that you can use via the right click menu.  Not super helpful as the main exporter because it is so slow and kind of buggy, but it has some great helper functions for adding some pieces back to the .vrscene file that the RT exporter creates.  In Maxscript what you need to do is:

1. Export the .vrscene file using ‘vrayExportRTScene’.  There’s a link above for more info.
2. Open the text file that creates and manually add the elements you want to the end of it.  You can use the functions in that vrsceneexport.ms file as helpers and examples.

Here’s the code I came up with to add the MultiMatte and VRayDirt ExtraTex elements.  I think it’s pretty straightforward, a real explanation is a little out of scope for this post.  Message me if you are having troubles.

--these are defined in the pre-installed vray script called vrsceneexport.ms
global writeRenderElementMultiMatte
global writeRenderElementExtraTex
global printVRayDirt
global printProxy

--make sure VRay RT is set for active shade, otherwise the export function doesn't work
if (findString ((classof renderers.activeShade) as string) "V_Ray_RT") == undefined then (
    messageBox "You need to set the Active Shade renderer to VRay RT for the export to work"
)
else (
    vrayExportRTScene vrsceneFileName

    --now add some custom bits through maxscript that aren't exported by the built in exporter

    --these export functions are provided by vray, should be in a script in the startup folder
    local vrayRE_cnt_MultiMatte=1
    local vrayRE_cnt_VRayExtraTex = 1
    local rem=MaxOps.GetRenderElementMgr #Production
    numElements = rem.NumRenderElements()
    local tempStream = "" as stringstream
    if (rem!=undefined) do (
        for i=0 to (rem.NumRenderElements()-1) do (
            local re=rem.GetRenderElement i

            if (re.enabled) do (
                case (classof re) of (
                    MultiMatteElement: (
                        writeRenderElementMultiMatte tempStream re vrayRE_cnt_MultiMatte
                        vrayRE_cnt_MultiMatte=vrayRE_cnt_MultiMatte+1
                    )
                    VRayExtraTex: (
                        writeRenderElementExtraTex tempStream re vrayRE_cnt_VRayExtraTex
                        vrayRE_cnt_VRayExtraTex=vrayRE_cnt_VRayExtraTex+1

                        if classof re.texture == VRayDirt then (
                        --built in exporter doesn't save vray dirt, so figure out what the name of the element is supposed to be and fill in the missing map info
                        seek tempStream 0
                        while eof tempStream == false do (
                            ln = readline tempStream
                            if ln != "" then (
                                tokens = filterstring ln "="
                                --print tokens[1]
                                if trimleft tokens[1] == "texmap" then (
                                    --print tokens[2]
                                    texname = filterString tokens[2] ";"
                                    texname = texname[1]
                                    seek tempStream (tempStream as string).count
                                    printVRayDirt re.texture texname tempStream
                                )
                            )
                        )
                    )
                )
            )
        )
    )
)
 
--now append the tempStream to the .vrscene file
if tempStream != "" then (
    outFile = openFile vrsceneFileName mode:"atc"
    format "%" (tempStream as string) to:outFile
    flush outFile
    close outFile
)