|
16
Jun
|
Building an Ikea Helmer Render Farm |
We need it yesterday. We always need it yesterday. A common theme in post production is that high-end creative must be produced on a tight deadline. Few things are more frustrating than waiting on an After Effects render while the clock is ticking loudly in the background. Like many post-production facilities, DotLot generally has a limited amount of time to deliver beautiful videos augmented with amazing graphics. Even though our computers are fairly powerful, we use of dozens of video layers and plugins at once, which taxes our poor little 8-core processors to the edge of their capabilities. By putting our heads together with our amazing IT tech Aziz, we were able to construct a solid render farm at a fraction of the cost of most high-end professional solutions.
For the uninitiated, a render farm is a group of networked computers that help to speed up render time by sharing the workload. For instance, a 30-second commercial has 900 frames. If each frame takes 2 minutes to render, then a single computer would require 1800 minutes, or 30 hours to complete the job. With multiple computers working together, each one can tackle a single frame at a time and then take the next frame in queue when it is done. If you have a five-node farm, that very same render would be completed in one-fifth of the time, or just 6 hours. In a world of looming deadlines, this makes a huge difference.
Our render farm journey started at Ikea. Sort of. A while back, Aziz sent me a link to a blog post about a guy who had built a multi-core Linux cluster using DIY parts and an Ikea Helmer cabinet. (To be fair, Obscured Clarity actually credits this guy for coming up with the original idea.) Prior to seeing this post, we had assembled a five-node render farm using old workstations that were left behind when systems were upgraded. It was a decent solution, but having five big Dell towers in a small edit bay made a lot of noise and heat and wasn’t very practical. This Helmer idea seemed to be a great alternative - we could build cheap, bare-bones systems; connect to them through our network; and they would take up less space than a single tower. Good deal!
Our plan was to split up the work - Aziz would set up the computers while I modded the case. His first task was to order all the parts for the computers. Since this project was somewhat experimental, we decided to start small with three render nodes, figuring we would add more later. Here was our buy list from NewEgg.
We originally ordered 3.5″ drives - DON’T DO THIS! The Helmer is only big enough to fit 2.5″ laptop drives. We returned the 3.5″ drives and ordered up some spiffy Western Digital Scorpio Black 160GB 2.5″ hard drives.
With all the parts in hand, Aziz set out to build the master drive, from which we would clone all the others. He installed Windows XP and the After Effects render module and then added all of our plugins and fonts. (We primarily use Red Giant’s Trapcode Suite and Knoll Light Factory.)
While he worked on that, my first task was to head out to Ikea to buy the Helmer. It’s available in a few colors (silver, white and red) but since we were building a render farm I decided it would be appropriate to go with the barn-red version.
Like most Ikea stuff, the basic build was pretty easy. The hardest part was cutting holes in the back of each drawer for the PSU and cooling fan.
Word to the wise - if you’re using a Dremel, be sure to use an appropriate routing bit for metal. I burned through a few wood routing bits before I finally realized that there was a bit made specifically for metal routing.
Once the rear panels are cut and the screw holes are drilled, you can mount your PSU and fan.
All that’s left to do after that is to slit the label holder on the front panel of the drawer so that you can fold it back to allow for a little extra airflow over the CPU.
The next step was to build the three systems into their drawers. Aziz used Clonezilla to make identical copies of the master drive.
In order to avoid issues with the bottom of the motherboards touching the metal surface of the shelf, we cut insulating sheets out of the tops of the motherboard boxes. High tech, I know!
With a handful of zip ties we were able to get the cables under control without having to cut any of the extra ones away.
We added a switch so that we could connect all of the systems to our network.
And our Pièce de résistance is our patent-pending screwdriver power switch. We had planned to order some lighted power switches that we were going to add to the front of each drawer, but there was a problem with the order and we needed to build the farm, so we went super low-tech and decided to forego any physical switch at all. We literally said “screw it” and picked up a nearby phillips head driver instead.
In this shot you can see the CPU fan is finally up and running…
We assigned each of the three render nodes a unique, static IP, and using the Windows Remote Desktop Connection feature, we were able to connect to each one from our primary edit system.
It worked like a charm!
Now all of our renders are three times as fast as they used to be. In the near future, we plan to expand our farm to a full six systems. Eventually, we might even add additional Helmers to make our rendering lives even better.
–Mike
Hey man I am currently wanting to build one of these for animation and special effects if you have time let me know if i am missing anything
Seagate Momentus 5400.6 ST9160314AS 160GB 5400 RPM 2.5″ SATA 3.0Gb/s Notebook Hard Drive -Bare Drive x 10 38.00 each 380.00
AMD Phenom quad cor am2+ socket x 10 70.00 each 700.00
JetWay JM26GT4-A-LF AM3/AM2+/AM2 NVIDIA NF6100-430 (MCP61P) Micro ATX AMD Motherboard
40.00 x10 400.00
G.SKILL 1GB 200-Pin DDR2 SO-DIMM DDR2 533 (PC2 4200) Laptop Memory Model F2-4200PHU1-1GBSA
19.00 x10 190.00
TRENDnet TEG-S224 10/100Mbps Ports + 1000Mbps Uplinks Copper Gigabit Switch
89 x 1 89.00
Each drawer a total of 10 drawers
1 160 gb hd
1 quad core processor
1 jetway mother board
1 gb of memory “will this be enoughmemory”
An open source render q software for maya from creative crash
Also fans and heatsinks as well
Total cost 1759.00 plus cost of fans and heat sinks
i am wanting to do serious effects for particles, fluids and such also do you know why vue takes 17 hours to render a single frame?
does this seem like overkill to you ? and from a software point of view what else would I need , I understand you are busy let me know what you think and thanks : )
Hey guys,
Can this system render Vue Xstream, Realflow, Cinema 4D, After Effects and from my mac pro?
thanks,
Christian
Hi Jeff,
Thanks so much for your comment. I’m glad to see that you’re taking this to the next level. I’m not familiar with many of the specific components you mentioned, but to answer your question; no, I don’t think there can ever be such a thing as “overkill” when it comes to building a render infrastructure. Even if you manage to outdo your needs for today, the new plugins and programs of tomorrow will soon make your render farm seem slow. Build it as big and as fast as you can within your budget.
In my experience, it can render anything for which it has the render modules installed.
hi mike,
i am also building a helmer render box … but i have a problem with the ethernet cable … the connector is too big … how do you solve this problem? or is your motherboard small enough
thx, stefan
@Stefan - The mobo we used fits perfectly in the drawer with a standard ethernet connector (the cheapo clear plastic kind) but there’s only a couple of mm to spare. If you have a bigger mobo, or a bigger connector, I could see where you might run into trouble. You might try cutting away the cable casing so that just the individual strands protrude from the back of the connector. It depends how much room you need to create, I suppose.
thx for the suggestion, i will try it …
Nice the helmers a great bit of kit just finished mine here’s a vid of it http://www.youtube.com/watch?v=X5bQmHpGNjU
What kind of heat sink/fan did you use on the motherboard? Im building my own render farm and all the fans and heat sinks I find are too tall for the drawers.
@erik - We used the stock fans that came with the CPU. We were rendering, but not overclocking, so we didn’t need any special cooling.
Hey thanks for the blog
I’m building the same thing and have a similar issue as erik. (i got the biostar n6853 motherboards at a great price)
Is it super important to have a fan on top of the processor? I bought 60mm silenx fans for the back, but what if i also added one to the front…. do you think that would be enough air flow to not burn up the processor???
Anyways id love to hear what the solution would be or if theres a cheap fan that would fit.
@Ryan - I’ve never run a processor without a fan, so I don’t know what would happen. But if you have the $$$ or an extra processor lying around that you don’t care about, then let me know what happens.
But… I think that having a fan on a processor has a big effect of pulling heat directly off the CPU, so I don’t know if it would work without some sort of cooling unit in contact. BUT… I’m no expert. Let me know if you learn anything new or interesting.
Maybe you have an answer for this and maybe you dont. Im still working on picking parts to build my own render farm. How important is hard drive transfer speed? Im looking at one hard drive that’s usb 2.0 and transfer speed is 480mps, but i’m also looking at on that’s usb 3.0 and its transfer speed is 5gbps. Will the usb 3.0 make rendering any faster? I know rendering is done by the cpu and ram is important but how important is the hard drive speed?
You’re right that rendering speed is largely driven by CPU/RAM. The hard drive speed would only be a factor if your system was rendering so fast that the hard drive write speed couldn’t keep up. I’ve never know this to be an issue with my renders, since the work we do tends to involve lots of layers/effects and thus is highly processor intensive.
How did you go about feeding in the ethernet cable through the drawer and into the motherboard? Did you drill a separate hole in the back of the cabinet or did you feed it through the opening you made for the power supply?
The back of the Helmer isn’t covered, so we just fed each ethernet cable through the top of the drawer that it needed to go into.
I’m having some problems with the networking. Did you have to create a homegroup or some sort of networking over an internet connection to get the render farm to work?
Also have you had or experienced any problems with the nodes overheating?