Website powered by

Rendering Pipeline - 2021

12/20/2021

Rendering Pipeline

I have been very fortunate in the past few years of working with not only talented Engineers and Programmers, but most importantly, generous and kind enough to support my desire and eagerness to learn more about how what I do on a computer is actually generated, rendered, in form of pixels.

This endeavor started many years ago with the goal of understanding a bit more how PBR materials work, so I could do my work better and more accurate.

During my time at Naughty Dog, I worked with Yibing Yang and was amazed by how she implement the cloth and hair shaders on Uncharted 4. I had close to zero idea of what she was doing, but remembered being intrigued by it.

At Sony Santa Monica, I had the pleasure of working with Florian Strauss, responsible for developing the entire Material System and Editor. I learned a ton from a software development and user experience point of view. Unfortunately, my technical knowledge at that time was very superficial so I couldn’t really understand the shading models or the rendering features we had available. So I focused on what was exposed, and the logic behind composing those materials.

During God of War 2018, I worked with Rupert Renard on designing and developing the Dynamic Material system on Characters, that created runtime blood, dirt, water, poison, snow effects across all of the characters. I was responsible for the technical part behind the characters, like sharing uvs and material properties, but on the coding side I had very little involvement, just participated on a more high level/front end type of contribution, defining specs and pushing the feature forward.

On my last year over there, I had the pleasure of working super closely with Doug Mcnabb, which was in charge of rebuilding the Material Editor. At that point, I was starting to study Unreal and play with shading network, constructing materials using nodes and very simple logic. Doug was always super kind on explaining the logic behind simple functions and even tried going a bit deeper on certain features like retro-reflectivity, but it was still too abstract. I could follow the very surface of those things.

I ended up moving gigs and started as a Character Director at Striking Distance Studios, at the very start. I was in charge of building and developing the entire character pipeline, and ended up helping shape our current Material Editor as well.

A few months later my start, Jorge Jimenez joined the studio as the head of Creative Engineering. I remember reading Jorge’s Activision papers many many years ago and I couldn’t be more excited to get to meet and work with him. Most of the skin shaders nowadays and eye renderings in realtime are based upon his studies and developments.

Jorge has been incredible kind with me for the past two years. We have a super tight relationship at work and we are pushing things. Can’t say much but I definitely need to thank him for all the help and time explaining how things actually work. I was able to fully start to understand the more behind the scene cg rendering, which led me into this new journey here.

I feel like I’m writing an essay here, but if you are reading it till now, great.

My relationship with programming started a few years ago with Python. Took some very basic classes on Udemy on Python for Maya - https://www.udemy.com/course/python-for-maya/

That gave me the basic introduction to scripting and since then I was hooked up. I try to write little snippets in maya as much as I can, and was able to write tiny batch actions for Unreal as well. Very simple stuff but definitely fun and productive.

If you saw the project before this one - I started building a full facial rig from scratch and I knew I could use ARKit to drive the rig. There are many live link apps already out there, but when I started looking into how to implement my own facial rig on my own app, I was fully stuck. I was able to download git projects and Apple sample code projects and replace the geo, but I couldn’t do absolutely nothing else.

So I decided to learn the language (Swift). I started with an Apple app called Playgrounds. Did all of the exercises there, more than 200. It takes you from very simple functions to writting very simple algorithms. After I finished all the exercises, I had a good grasp of the very basic language and very very basic logic, but I had no idea how the frameworks were structure and how to implement them. So I started following online tutorials, such as https://www.youtube.com/c/realityschool . I learned a lot too but still felt like it was too advanced, so searching for more courses like from zero to hero type of class, I stumbled upon Tito Petri website - https://hotmart.com/product/en/aprenda-programar-com-tito-petri and joined his school. I’m going through the entire AR path and built around 8 example apps already. That was exactly what I was looking for. Totally worth mentioning too, Tito was my first 3D Instructor an