Making of: "The Scientific Method"

Anyone who's seen any of my previous videos will know that I have a habit of making life difficult for myself. I tend to try new things which seem simple on the surface but end up becoming technical nightmares, yet with a project time of almost 3 months, this episode took the biscuit..

The previous "most difficult" video before this was my entry to the [Ctrl][Alt][Space] competition - a 4 minute video called "Mining Class" - the sheer amount of Pre-rendered CGI I composited to that video was ridiculous.

It had several CGI actors, multiple camera angles, Lip Syncing and even a fully animated female character - complete with Stroopwafels - it also had an orchestral classical soundtrack which I licensed (although it still got flagged several times by YouTube's content ID system - all resolved now).

While I was building Mining Class, I ran into technical issues with the Daz Studio renderer. It happened as I was exporting the rendered characters. Strange glitches would appear in the footage - possibly as a result of exporting multiple scenes at the same time using multiple instances of Daz Studio in parallel to save time so they could render a camera angle each. It caused major headaches.

In short, the process of exporting the images was taking too long and it was unreliable. It also meant that only static shots were possible so getting the shots accurate within Daz studio was paramount and had no room for mistakes in positioning or lighting.

I needed another way.

The story I had in mind also had a requirement for a central room which could serve as both a set-up room and a room for the punchline. I wanted to show the sheer size of the Lakon Type-9 in all its glory. What better way than to have a control room with a hologram map, vending machine and 4 chairs. This would serve as a nice common room; a staple of Sci-fi comedy. Nothing in-game existed for this, so I would have to put a scene together using 3D models.

My work on Speed meant that I knew I could make external environments using models in Daz Studio - but for performance reasons, they had to be pre-rendered. I wanted the scene to be a lot more fluid with a moving camera and a lot better lit. So pre-rendered was out of the question.

Then there was the subject of the Hologram.

It's no use having a hologram projector if there's no hologram to display.. I really wanted to have some form of presentation style plan exposition - something out of Minority Report, the X-Men or Star Wars.

I toyed with the idea of using game footage and fixing the camera angle while I spun a video and masked it.. but it'd look bad and I would be limited with my camera. I wanted flexibility.

Element 3D to the rescue!

I discovered a really powerful plug-in for After Effects by the guys at Video Co-Pilot called Element3D. The plugin cost around $99 and it looked like it could do a really good job. It was really fast and used modern shader technology. It's renderer knocked the pants off the Daz Studio renderer both in terms of performance and quality - also it integrated seamlessly into After Effects, So I figured I'd invest and get it; After-all, I plan on making videos as long as people enjoy them.

What this gave me was lightyears ahead of what I had before the render quality was brilliant and like a kid in a candy shop, I was spoiled for choice. I wanted to do everything with it.

The render problems which had dogged my previous episodes were gone in an instant. No more rendering series of pngs of heads. I'd be able to use the real thing and light it within the scene properly and adjust everything to better match the captured game footage.

Furthermore, the issues of the pixelated fibreglass hair from the Villainous would be gone and the shaders which come with the plugin would allow me to do a much better job of making the characters' skin look more realistic - as well as metals, plastic and rubber.

I'd still have to animate the characters in Daz Studio though and export the animations because at this moment Element3D doesn't support BVH files (@VideoCopilot - Please include BVH support - it'll save me so much time)..

So I was back to exporting sequences of frames, but this time they'd be geometry - not PNGs so I'd have a lot more flexibility about camera angles, lighting and scale.

The problem when you get a new fantastic tool is that it's easy to overlook the faults.. either in the software or in the process. I would soon encounter massive roadblocks.

To export the frames, I had to use a 3rd party script... and the files it generated were.. to be honest, ENORMOUS. For some reason - I was generating geometry files in obj format which were between 80 and 120mb each. When you consider that this was for a single character and some of the sequences were 2500 frames each. I had 2 characters in most scenes, 3 in others - all CGI. The total amount of drive space required was over a Terabyte and then there was the amount of time it took to export.. some scenes took 5 hours to export the frames. My poor hard drive.

This is nothing however, when it came time to render a test which contained the animation, the estimated render time was over 120 hours. Not really the best use of my time...

Decimator to the rescue

I had to do something about this. Amazingly, Daz studio has a plugin for $99 called the Decimator. This allows you to reduce the number of polygons for each of the actors and props down to a more reasonable amount... from over a million down to tens of thousands. And from 800 thousand polygons for Ascorbius' hair, to 30,000 - it was a significant saving. In some cases I was shaving 100mb per frame... the scenes with the decimated actors loaded a lot faster and it meant that the rendering was now down to a much more manageable 3 hours... for a 12 minute video.

Native BVH would be faster still however...

This sounds like I'm complaining, I'm not. The new features meant I could do a lot more stuff... especially with that hologram.

Making the Hologram

The hologram was a technical challenge I'd never dealt with before. How on earth would I make a 3D model from the Ancient ruins - a model I could spin in any direction and superimpose a schematic of the underground complex.

I had no way to get the mesh from the game, I'm not sure if RenderDoc can even pull the geometry however, I also needed the texture so it looked correct from all angles.

I found the answer in a piece of software by Agisoft called Photoscan which I was pointed to by CMDR TIKAS (who's videos are amazing, I hold their quality and presentation in very high regard ). I used the standard edition - which ran at $179.

The great thing about AgiSoft Photoscan is that it can take a series of photos of an object and make an accurate 3D model out of them. So I flew around the ruins site in a ship-launched-fighter at a distance and captured footage. I then extracted images from this footage and imported them into PhotoScan... it then made a really nice model of the ruins.. Too Nice. I had to decimate it using the built in tools to make the model a reasonable size, export it and then load the exported model into Blender and trim it down to a nice circular shape.

I then had the task of making the underground structure. I'm still not very good with Blender, so I resorted to my old trusty friend "MilkShape". That let me make a really simple low poly object which could represent the innermost structure of the Guardians site.

I knew I wanted to do something with the actual Data Obelisks, so I placed a 3d shape over each of them - I wanted the location to be as accurate as possible in terms of their position and their elevation.

With the hologram model done, I started to consider the animation. I had 2 characters. They would be sitting for the most part, one would be animated by the game and the other would be animated manually. However there were scenes in the map room where the actors would have to interact with each-other while delivering their lip-synced lines.

Lip Sync

Lip syncing would be done with the Mimic Pro tool within Daz as usual; Playing back the recorded audio for a character at high volume so the Mic can pick it up and animate the character.

It's a little clunky and inaccurate, but for the purposes of the video, it would do.. however I will be looking for alternatives pretty soon.

MoCap

For the standing animations I figured I'd try my hand at Motion Capture - trying to breathe a little more life into the scene.

MoCap is normally something which is out of reach for the average movie maker.. typically exclusively the realm of movie and game studios. However, options were available,

A piece of software called iPiSoft MoCap Studio3 claimed that it could use a Microsoft Kinect and perform "Markerless Motion Capture" by analysing the depth map and footage from the Kinect, allowing it to work out where the actors arms and legs are, and act accordingly. So I got the software.. and a Kinect. I was mostly happy with the results... mostly.

In reality, MoCap is still very very tricky. The software often gets confused when the lighting isn't perfect or when the actor isn't quite close enough.. and does a kind of "Riverdance" with the limbs in ways which would be impossible. But with a little persuasion, the animation can be tweaked to be more realistic.

Once the MoCap sequence is complete and any desire to dance a jig has been removed, a Daz studio skeleton can be imported, the bones matched, exported as a BVH, imported into Daz Studio, merged with the character - which then animates the character. It can then be exported as the sequence of OBJ files for Element3D in After Effects.

Some scenes just turned out to be too tricky, so I either KeyFramed them or used AniBlocks from Daz Studio.

Suit up

Most of the scenes use the existing player flight suit, which is lucky for me as I don't have to model a flight-suit.. well, I didn't have to. This episode however has Ascorbius walking about in his flight suit - which looks nothing like any of the suits I had access to. So I called on the re-skinning talents of Turjan Starstone who took a series of high resolution reference photos I took from the HoloMe editor and stretched, skewed, rotated and generally beat into shape to match the texture file I needed.

The end result after some additional tweaking to remove any colour problems and to work out the bump, reflection and light maps was a pretty passable flight suit.

So many thanks to Turjan for that.

Filming and Health & Safety.

There were actually physical challenges to virtually filming this video. I nearly suffered an injury while trying to get footage of the topmost left hand corridor of the Type-9.

The Type-9 has the most amazing bridge of any ship in the game. Sure, some ships have a larger, more spacious bridge where you could have a party, host the Olympics while still having room for the Circus, but the Type-9 has this industrial horror show of a bridge. There are dark nooks and crannies which could possibly hide all sorts of untold nasties.

I've tried to get a glimpse beyond the corridors of the bridge before and failed. I always film using the HTC Vive, it gives me an incredible amount of flexibility for shots and allows me to reach places normally impossible... the top left corridor being one of them.

The only way to do this is to place the headset onto the floor at the extreme right of the room (while staying visible to the lighthouses) - as far as possible and then to reset the camera position. This tricks the Vive into thinking the player's head is somewhere else. So then you can stand upon a chair on the opposite side of the room and capture video from a completely different and normally inaccessible vantage point. With my vive in my hands, arms extended and turning to look at the monitor to check the captured footage while standing in socks on a wooden chair - there were a few almost tragic moments.

Thankfully In the room I was doing this in, the largest RoomScale capable room in my house, I was able to get the footage - but being careful because if the headset went beyond the view of the lighthouses, the ruse would be up and the vive would automatically reset the position.

A similar trick was used to get the shots within the fighter. We've all seen the shots you can get using the built in camera angles for the camera suite, but no-one really gets to see the side of the pilot's helmet,

or shots just outside of the cockpit. These were all achieved using the technique highlighted above.

Jayne and Scorb aren't the only characters in this video who needed animating. There was also the cold blooded alleged agent from Canonn Interstellar. His character had additional technical problems in that he already had a head. I didn't record his footage in the standard way. I was thinking I'd be able to use my lip-sync plugin within After Effects - however after a while it became apparent that the effect would look .. goofy. So I made a head, erased as much of the background head as possible and placed the new head over the top.

Special Effects

I added some smoke and sparks to the scene to let everyone know that his ship was in a really bad state. These were done by using Video Co-Pilot's Action Essentials elements and then with a combination of 3D positioning and masking, I placed them in front of and behind the pilot's chair. In fact the elements would be useful throughout the episode.

CGI Characters

There were 2 more completely CGI characters in this video which would be introduced as a surprise.

The first was Betsy. She's a 500 year old diesel powered drill tank. She's fast and powerful - in spite of her age. And in true classic style, She's is adorned with a nose-art decal.

(I've no idea who the original artist was but thank you for making it, it's glorious)

In fact the only problem with Betsy in the story is that Diesel is really difficult to find in 3303. Hence Jayne's concern about running out.

The model for Betsy was an asset on the Daz Studio store which I retextured to add a few details I wanted,

it encompassed the feel I was going for perfectly. So I exported the model and was able to bring it into Element3D easily. The only problem was that Daz and Element3D have different ways of animating wheels. Daz uses bones and Element3D uses a material group system. This meant that without considerable work, I wouldn't be able to move the wheels or the blades... But if you didn't notice that when you watched the video, it didn't matter ;)

Betsy's movement over the terrain had to be simulated by watching the video and keyframing rotations and translations within After Effects. It's a little clunky, but it does the job.

The other character is the Guardian creature named ROG.

He's a difficult character because his physiology doesn't completely match a human. He's a Daz Studio character from the store - the closest I could find for a alien matching the guardians description - but he's not a perfect match. I wanted a scary looking creature which could also look friendly.

So I wasn't able to MoCap his actions. So it was down to Key-framing him for the entire time he was visible.

In the show, Rog has a Red colour and glowing eyes, these were things I was able to achieve within Element3D pretty easily. His animations also had to be exported as frames and imported into Element3D as a sequence.

But I'm really happy with how Rog turned out.. and I'm pretty sure we'll be seeing more of him in future.

Magic

Towards the end of the episode, there's a Magic: The Gathering game being played. Somehow Ascorbius taught Rog how to play magic in a really short time and Rog is now defeating Scorb with a Blue control deck.

To do this, I had to take a screenshot from the PC game Duels of the Planeswalkers, perspective correct the image and then load it into Milkshape to make the cards.. then apply the image as a texture to the cards using the UVW editor.

This took a little while, for a 5 second sight gag.. But it was worth it. I was able to export this as a OBJ file for use within Element3D and place the game on the scanner table.

Finally, there was the music.

After the licensing problems I encountered on Mining Class, I decided against using licensed music - even classical - I wanted a score composed for the episode. Composing music is challenging enough, Scoring music is even harder. My brother, who produces music under the name of S1Studios was tasked with coming up with around 12 minutes of original music to suit the mood of the show - working from a very early draft of the episode. I'm very happy with what he produced and feel that his work made the show feel a lot more epic.

So this was the 100 mph view of how the episode was made. I hope you enjoyed it,

I'll be improving my workflow for the next video, so it shouldn't take quite so long.

Have a good time, and Fly Aimless

Ascorbius