LRCHS – Projection Mapping – 1st Post

The 90th anniversary of the opening of the Little Rock Central High School building and the 60th anniversary of the Desegregation Crisis are coming September 18-25, 2017. It will be a week of activities that commemorates the anniversaries and culminates in an event that features a projection mapped animation on the façade of the high school building.

This first blog post is about a major milestone for the animation, which is a completed virtual 3D model of the facade including its four statues. Now that the model is complete we can finally get to work. The majority of the animation we create will be based on the architectural structure of the facade. I can’t believe February is almost over! It took me over a week longer than I expected to finish this phase of the project due to distractions including an illness that caused horrible headaches as well as external issues and projects and some personal goals beyond the projection mapping project. Hopefully the headaches are past – I can manage the rest.

Here’s the basic model:

4statues

We can add lighting that can make it appear as if we’ve hung actual lights near the building:

spotlights

We can also play around (this is just a test and not final imagery):

lightjade

And add stuff:

1927

Here’s what it should look like at the campus. We intend to add some lighting around the central facade as well.

projectiontest

The Facade

The limestone part of the high school’s main entry has several nice 1920s Art Deco details and is sculptural in nature with deep set doors and windows and jutting pedestals for the four statues. I still need to add the letters for the statues. We will hopefully be able to temporarily cover the windows and doors so they won’t be so dark. We will also need to cover the lanterns so they will reflect the projections.

dsc00432

Ambition, Personality, Opportunity, and Preparation

When facing the building the four statues from left to right are Ambition (male), Personality (female), Opportunity (female), and Preparation (male).

I’ve been told that the four statues were “ordered from a catalog” and not unique to the building project. Their body styles are reminiscent of Michelangelo sculptures with their long muscular arms and Greek facial features. Preparation must have been the sculptor’s version of David – see his contrapposto stance, physique, lowered right arm (holding a scroll in this case), raised left arm holding a book instead of a sling, and a left-facing gaze.

ch-interior_110 512px-27david27_by_michelangelo_jbu0001

Their dress is based on ancient Greek Chiton. The sculptural style is “wet drape” where the cloth clings to the skin to reveal the figure’s body underneath. This is most obvious in Preparation with his torso that practically looks bare, and you can see it in Opportunity as well. I modeled these statues by starting with nudes so I could get the wet drape look right.

I think later blog posts will go on another website dedicated to this project. Geeky stuff will stay on this blog though.

Geek Stuff (most of you will want to skip this)

I modeled the facade by building basic geometric shapes and aligning them to a photograph I took last summer. I actually got most of this model finished by last fall. In January I added the smaller details and lanterns.

The statues were very time consuming and I knew they would be… I downloaded a few nude “base models” from Blendswap, which are designed to be a starting place for creating a character. For the females, I used the body of one and the hands and head of another. After splicing them together I pushed and pulled and extruded faces, edges, and vertices to make them match the sculpture. I also used sculpting tools to smooth and guide areas of the model. The models are considered low-poly, which makes them easy to animate and handle in the 3D software. When they are rendered they are smoothed using Pixar’s subdivision surface technology. It turns a blocky mess of polygons into flowing garments.

For the capes I essentially started with a line and extruded it and moved it to create the overlapping folds. For smaller details I just cut the larger polygonal faces into smaller ones that I could then push, pull, and sculpt into their final form.

Once a model seemed ready to go I aligned it with the main photo of the facade. I had closeups of the statues to do most of the work, but since the photos were taken from below, the proportions were not accurate so aligning with the main photo was key to getting them the correct overall size. Because of the proportion issues and a number of other things, I modeled them just looking at my photos rather than trying to align them to photos in the 3D viewport, which is common for character design.

While modeling, the virtual statue is standing in a T-pose. I used a T-pose because we will most-likely apply some custom motion capture animation and our motion capture system (Perception Neuron) requires a T-pose to start. Another common starting point for a character model is an A-pose, which is more relaxed, but not a good idea for our purposes.

After getting the proportions correct I added a skeleton to the model. The skeleton is based on the needs of the motion capture system. The model is bound to the skeleton so whenever I move a bone, the model with deform with it. I used the bones to pose the model to match the statues. I actually animated the movement so I could go back to the T-pose easily as well as test the model deformations as the bones moved. Some of the dress is not driven by the skeleton at the moment. That will come later via cloth simulations.

opportunityposing

I modeled the statues this way because I knew we would be animating them and they needed a structure that would support animation. A more accurate alternative to modeling by eye would have been to scan the actual sculptures. Scanning could be done via LIDAR, but would have been prohibitively expensive. Or, it can be done with lots of photographs from multiple angles via photogrammetry. Shooting the sculptures with a drone and extracting frames from the video would have been a way to get the images needed.

The upside to scanning would be a very accurate model, but there are downsides. One is that the scan would have to be retopologized, which can be time intensive, to make it animatable. Another is that the models would not have a backside and the arms would be stuck to the bodies so they would need hand modeling to create the back and make the arms free. I would have been up for these things had they been scanned last fall. Unfortunately they are 22 feet above the ground so logistically it is not a trivial issue to get to them.

From here it is a matter of lighting, creating cool surface materials, animating the statues, opening the doors, or whatever else we come up with. Even things that don’t directly change the facade, such as showing a photo, will be rendered against the virtual facade so the photo will appear to interact with the building.

Blender

screenshot

I used Blender to do all of this work. It is just a joy to use. Some things that came in handy (these aren’t necessarily unique to Blender BTW):

  • Use photos as a background in the camera viewport to help create a 3D environment that is similar to the size of the actual building
  • Changed one of my 3D panels into an image viewer so I could have a photo of a statue up at all times.
  • The Shift Key – I use a Wacom Intuos 4 Medium when working with graphics software. It has a bad habit of moving during a click or not really making the mark you tried because it was so small. When changing a parameter in Blender (practically no matter what it is), you can hold down the Shift Key while doing it and it will increase the accuracy of the parameter by not allowing it to change drastically no matter how much you move the stylus. I can make big movements to do small changes. BTW, some graphics programs do have a similar function, just not all…
  • Matcaps – haven’t really used them before, but they make modeling organic forms much easier. They allow you to customize how the model is shaded in the viewport so you can see the curved surfaces easier.
  • Proportional Editing – Used when moving a vertex or small group of vertices and wanting surrounding vertices to move with them, but not as much. Super helpful when making proportion changes or needing to move parts of the model around to accommodate the posed body. Especially useful is the “Connected” mode where it will only move vertices connected to the one you are moving rather than ones that are just nearby. You can also change the falloff to control how the other non-selected vertices will change. BTW, this works on more than just vertices, just using that as an example.
  • Subdivision Surfaces – Blender can show the subd effect while editing the model either by showing the base model and the smoothing separately or by bending the base model’s edges along the surface of the smoothed model. This really helps know how the changes of the low resolution model will change the smoothed model.
  • Solidify modifier – I made the capes a single polygon thickness and used this modifier to give it dimensional thickness. When sending the models out to Jim and Jonathan, who use Cinema4D and Maya, I will “Apply” this effect to make the geometry permanent.
  • Cycles with two GPUs – it’s so fast! To do test renderings and make the images in this blog post it is amazing how fast Cycles can be. The images here took about a minute and a half to render each one. It’s also crazy easy to make objects into light sources. I do most of the work on my iMac and then switch over to my Linux computer for rendering.
Advertisements

Pipeline Tools

The impetus for this post comes from two sources. One was an interview I watched on fxguide.com with the head of the Discreet Logic products at Autodesk (Smoke, Flame, and Lustre – kind of), who talked about Flame and Smoke being finishing tools rather than pipeline tools. He was talking about Nuke and how in many ways it is not comparable to Flame because Flame/Smoke are much more than shot-by-shot applications. Instead they are about seeing your whole project and bringing it to a close with titles, complex effects, color correction, etc. at realtime speeds so clients can be in on the process. So it got me thinking about what apps are used in the pipeline and which apps can be used to complete a whole project (animation or film). The other source was the recent news about Luxology and The Foundry merging. These companies create very different products and there are some pretty cool opportunities with the merger. One thing that came up, however, in interviews with users of their products is that The Foundry’s products are very expensive and out of reach for small studios and Luxology’s products are very inexpensive and easy to use. I was really surprised by the candor of John Knoll (ILM vfx supervisor) and Tim Crowson (Magnetic Dreams, Nashville) about how The Foundry’s tools are expensive pipeline tools that are hard to use and how Luxology’s Modo is low-cost, easy to use, and creates really high-quality output. John Knoll also seemed to say that The Foundry could learn a thing or two about software design from Luxology. Stu Maschwitz has also remarked about how there is a move towards easier to use software over never-ending software options that are hard to understand and rarely used. Even the new version of Maya, which is famous for its ability to make the easiest thing take as many mouse clicks as possible in multiple editors, has simplified its interface to make everyday tasks easier. For years I have argued that it should not be so hard to get work done in software applications. I always thought it was funny coming from me because I like a lot of complexity and I am very technically savvy, but I was finding that it was complexity for complexity’s sake or it was a leftover from the app starting out as a pipeline or in-house tool that grew to a more generalized application. Cinema4D is really popular with motion graphics artists because it is relatively easy to use and can produce high-quality imagery with a complete toolset. Blender is free, but it is shrugged off often because it’s free – how can it be good if you aren’t paying thousands of dollars for it?

There are a lot of software applications out there for creating digital content. Depending on your projects, production team size, level of quality, technical expertise, need for sharing data with other artists, GUI preferences, budget, and probably several other reasons you will find yourself looking at all the products out there and trying to decide how you should invest your overhead funds.

Two general options are out there for developing a pipeline. One is to keep everything under one application and the other is to break the process out to separate software tools. The primary reason to use one tool is simplicity – keeping all of your assets and workflow together in one environment. The main reason to break up the pipeline with separate tools is to use the best features of different applications to raise your production value as much as possible (assuming that each tool helps complete a task better than anything else).

An example process that can be done by one or many applications:

  • Live action post
    • Edit
    • Audio mix and sweetening
    • Color Correction
    • Visual Effects
    • Final compilation for delivery

Live action post from the edit to final delivery can be done in one application, such as Final Cut Pro, Avid MC/Symphony, Premiere Pro, and Smoke 2013 (among others, but these tend to be the ones that everyone likes to talk about). It is possible for one editor/artist or a small team to produce content at a high-level of quality in just one of these applications. However, for projects that require complicated sound mixing, extensive color correction, and/or complicated visual effects, a single application may not have all of the tools to do the job at the expected quality. Here’s how this would look if it were broken out into individual pipeline tools:

  • Live action post
    • Edit – Final Cut Pro, Premiere Pro, Avid MC or Symphony, Smoke 2013, etc.
    • Audio mix and sweetening – Pro Tools, Logic, Ardour, Audition, Nuendo , etc.
    • Conform – Hiero, Resolve, Scratch, Baselight, etc.
    • Color Correction – Resolve, Speedgrade, Scratch, Lustre, Apple Color (discontinued), Baselight, Magic Bullet plugins, etc.
    • Visual Effects – After Effects, Nuke, Digital Fusion, Flame, Smoke 2013, etc.
    • Final compilation for delivery – Avid DS and Symphony, Smoke

Using several applications creates some issues that the post-production team will have to deal with. The first is the ability to easily move data between these applications via XML/EDL/OMF/etc. or exporting from one and importing into another. This can be a headache if the applications fail to move data properly. There are also possibilities for data or image quality loss. The other big issue is cost. If the artist/freelancer, small shop, or institution has to maintain licenses of all of these tools then it can get expensive quick. Looking for bundles, suites, or free software often look better than purchasing from different vendors for each application.

3D animation and visual effects is even worse. There are several general animation and compositing applications that can do very high-quality work, but there are also pipeline tools that can offer better solutions to specific problems that can be tough for the general apps to do. Here are some steps in the workflow for doing 3D and/or VFX work:

  • 3D Animation/Visual Effects
    • Model
    • Rigging
    • Layout
    • Animation
    • Simulation
    • Tracking
    • UV Layout
    • Texture Painting
    • Shading
    • Lighting
    • Rendering
    • Compositing

Lots to do and lots of applications to do each part. First the general animation applications: Autodesk (Maya, 3dsmax, Softimage), Cinema4D, Lightwave, Modo, Blender, Houdini. There are others, but these get most of the press. Each can do everything listed below except for compositing (some can’t composite). For compositing just look at the Visual Effects step from the live-action list above and add Blender, Softimage, and Houdini.

  • 3D Animation/Visual Effects
    • Model – General 3D app, ZBrush, Mudbox, Modo, FormZ, Rhino, Inventor, Vue (for landscapes), Silo
    • Rigging – General 3D app + scripting, Motion Builder
    • Layout – General 3D app, Motion Builder, Katana
    • Animation – General 3D app, Motion Builder, in-house tools
    • Simulation – General 3D app, Naiad (recently assimilated into the Autodesk collective), realflow, Houdini, FumeFX, in-house tools
    • Tracking – PFTrack, Syntheyes, Blender, Mocha Pro, boujou
    • UV Layout – General 3D app, Headus
    • Texture Painting – Mari, Bodypaint, Photoshop, Krita, GIMP, etc.
    • Shading – Renderman, OSL, other renderer shading languages
    • Lighting – General 3D app, Katana, Keyshot
    • Rendering – Mental Ray, Vray, Renderman, Arnold, Maxwell, Luxrender, Appleseed (in early stages), 3Delight, Krakatoa, etc.
    • Compositing – After Effects, Digital Fusion, Nuke, Blender, Houdini, Softimage

A boatload of apps here… Moving data between these applications can be very difficult and some pipeline apps like Realflow and renderers require extra costs to render on more than one computer at a time. At facilities that use multiple pipeline tools and/or multiple general 3D apps there are usually Technical Directors (TDs) that create custom tools in Python and/or Perl or a native scripting language like MEL to efficiently move data between apps. This extra technical support is usually beyond the means of small shops, individual artists, and institutions so they tend to feel the pain when going between applications. How are things changing to help these problems?

  • Since Autodesk owns half these apps they have worked on getting data between each app easier – just have to upgrade each year to wait for better workflows
  • Large studios like ILM and Sony Imageworks have released software as open source to make data interchange easier. Projects include OpenEXR, Alembic, and OSL. Other open source projects like the Bullet physics simulator have been integrated into open source and commercial applications.
  • General 3D apps are getting better at what they do so extra funding for pipeline apps are not as necessary except in extreme situations. See the additions of sculpting to Cinema4D, physics simulators in Maya and Houdini, renderers like Cycles and smoke simulators in Blender.
  • Prices are falling and competition is good. The Pixel Farm recently lowered their costs, Luxology’s Modo continues to get better while keeping its price lower than all other commercial general 3D programs. New lower pricing structures for the higher-priced commercial applications seem to come out each year – or bundling like Autodesk does.
  • Open source solutions exist, such as Blender, Inkscape, Krita, Luxrender, Ardour, etc.

Do you really need the pipeline apps? Ask yourself a few questions, such as, can I afford it beyond my regular tools? will I use it regularly (ROI)? do I have a problem that can only be solved with one of these apps? do they play well with my existing apps? do you need to purchase a specific app for a specific position on your team, such as texture painter or colorist? Hopefully you will not find yourself saying no to being able to afford them, but saying yes to having a problem that can only be solved with one of these apps, but don’t be surprised when it happens. Also look for hidden costs, such as render licenses, limited OS support, and annual license fees. Consider leasing specialty apps when that option is available. More than anything – consider the need at hand and choose the tools that can get the work done at the level of quality you expect. Just because there is a specialty tool out there does not mean that it is the only thing that can do the work. BTW, I’ll try to add some links and costs in a followup post.