3D Stylized Character Modeling UE4 Production

Introduction

Some might already know about my old project – Exparia (a making-of article on 80lv was published last year). It was well received and I see that our communities have a huge interested in UE4 in-depth technique of stylized look dev and rendering to make 3D model. Therefore I am pleased to write more about this topic, in particular, a case study – my latest project – Pisciasa. She is one of my OC collections (girls with mecha monsters series) and above all she is my entry for the recent contest by CubeBrush – ArtWar2.

In the process of doing each of my projects, I have been working non-stop to train myself in both artistic and technical skills. I have taken part in Inktober to train my 2D drawing skills, I have written some more tools & scripts to aid with 3D production at Thundercloud. Also, I found some more interesting shaders technique in UE4 that could benefit the way I do the stylized rendering. I learn many things in the past months since Exparia and I want to put all of my skills and experiences to the challenge in Artwar2 using my latest work – Pisiasa, so in this article, you will have a brief look through all the techniques and workflows I have developed for Pisciasa.

Ambition

The biggest ambition of Pisciasa is the standard that I set myself to push the quality of the final 3d piece to look as close as possible to the 2D illustration concept. I want people unable to distinguish between the final 3D piece and its 2D concept work, thus improve the quality of the 3D in both art direction and production techniques. Once I can achieve this goal, then I would be confident that I can execute any of my 2D works into a decent stylized PBR work in UE4 without losing any of its feeling or artistic look in 2D.

Having experience from Exparia and my past projects, I know there are numerous of points I need to work on. First I need a more detail 2d concept so that I would know exactly what kind of look/visual goal I should aim for in UE4 as well as what kind of effects, stylized shaders I need to make in UE4.

The render quality of the final image in UE4 also needs to be improved, having a better shadow, better lighting, better AA, and details render etc.

Lastly, on top of all, I need a better planning and schedule to help me finishing Pisciasa before ARTWAR2 deadline. Which having a clear 2D concept as an art goal help me greatly in planning my production and vision the final result.

General view of workflow

The modeling & texturing workflows of Pisciasa are pretty basic and straightforward, for more detail of how I did the design and planning my modeling/texturing process you can visit the production breakdown article on CubeBrush blog which will be published soon by the time I am writing this, and process video below:

Fish modeling progress

Face Texturing

Body Texturing

Hair texturing

One notable point is that I improved my old base mesh for Pisciasa face & body model. I worked on my base meshes and articular topology and anatomy modeling over time after each one of my project cycle. Here are some images to demonstrate the features of Pisciasa base mesh, you can see how the topology in relation to deformation work during animation. If you are interested in Topology and edge flow for articular modeling (the relationship between Topology, anatomy, and joints position for best deformation result while animating). If you are interested in taking a deeper look into Pisciasa body, please go here.

Major improvements in shader development

One thing for sure that even if I could manage to carry out a spot on modeling and texturing, what really matters at the end that decides whether I could achieve the same feeling as the concept is a later part of the project – Shader Development and lighting. These steps will bring out more life out of the static look of stylized hand-painted models which are usually presented in plain unlit mode.

There are reasons why a lot of people refrain from adding lights and using no other shaders than unlit material to present stylized hand-painted texture model because the painted surface does not contain enough information for the light to behave like it should, thus making the texture look faked or rather just reveal the true self as static images being painted on the model. But if I can manage to develop a shader that somehow has lighting detail that looks like a stylized painting then I can pull this off.

Here ‘s a cheat sheet so you can have an overall view of Pisciasa shaders setup. Honestly I am no UE4 look dev shader expert, most of my shaders don’t really have fancy coding to manipulate the way they react to real lights in the scene (I really need to learn to code shaders in the future) but rather I manipulate basic things that mainly affect model’s surfaces like Base Color, subsurface color and normal in UE4 shader editor.

Setup for look dev

I used BP_light_studio as a starter setup for doing look dev which gives me a nice HDR dome, directional sunlight and other things like fog, skylight etc. Also, I can rotate the BP_light_studio actor to get different sun angle to test the model under different lighting condition (day/sunset/night)

I used a lot of fresnel and normal manipulation functions to develop my shaders so I always apply my shaders onto different basic shape to test how it reacts on different perfect simple surfaces, it ‘s also easier to troubleshoot and detect bug if there is something wrong going on with the shader.

Face & body skin shader

Face shader is always the first one I start working on, once I can have a decent look at the face I will then move my way up from there. I used the old Skin shader I made for Exparia as a base, I have adjusted some nodes and graphs for a better result in the fake reflection function nodes, detail breakdown of this shaders can be found in this article.

Below is a capture of the graph with clear annotation of each nodes group. I could use reroute pin to clear up the chaos looking graph but for some odd reason my UE has a bug when it can’t compile the output correctly when using reroute pins, so I apologize for the wild-looking wires.

Pisciasa Closeup Demo:

Fake Stylized Reflective material function

One of the key features of the skin shader I set up is the material function for stylized fake reflections. What it basically does is it takes whatever 2D cubemap texture your input (it could be a photograph, a hand painted texture, a concept art, etc) as reflective texture added on top of an existing diffuse input. The function also lets you control the fresnel amount, as well as the glow of the hi-light area as if the input is an HDR image. I used various forms of this function in many of my shaders to archive stylized look.

066 comparision withCustomReflection 1024x393 1

Different HDR or Cube Map inputs produce a different style of reflective material

New improved setup for stylized eyes material

Below is my old eye setup for Exparia, which is a bit difficult to maintain or to make any adjustment to the final look because it consisted of several different meshes with different shaders of different lighting models (opaque, cutout and additive translucency) which is a bit overcomplicated to maintain.

I came up with a new better setup for Pisciasa eyes, now her eyeball only consisted of 1 mesh, a simple eye sphere. The common problem with simple eye sphere is that they look really weird when viewing from side view angle because real eye’s iris is actually caved in and the area that curves out is just the cornea lenses so we have a very complex refraction effect happening in the eye area. To get over this problem, I used the Bump Depth node in UE4, which is used for parallax mapping, but in this case I will set it up so that the parallax will only happen when you view the eye at a narrow-angle (which is controlled by a Fresnel mask), this effect in combination with the fake reflection on top would fake the refraction effect really nice.

In principle, the eye shader is the same as my skin shader except that it has an extra effect – the fake refraction/parallax effect which I later integrated as an extra feature into my existing skin shader. Thus this shader also works well if your character’s face has texture-base eyes painted on ( not separated eyeball), you just need a depth map that masks our the iris area of the eyes for refraction effect only happen at these areas.

Stylized anisotropic shader

One of major improvement I have made for Pisciasa ‘s shaders is the stylized anisotropic (fake) shader used on her clothes and hair. It mainly derived from the custom reflection material function I made. Setup of the shader include 2 reflections effect overlay on top of Base Color pass, one at top area and one at the bottom area, these are to fake the effect of sun and ground reflections, I also have parameters to control the scale and position of these reflections.

067 anistropic material function 992x1024 1

A stylized Anisotropic material in action

Derived from the stylized anisotropic reflection function above, I have made a stylized hair material that utilizes this function and added some extra specific features for hair.

Stylized hair shader

This one utilizes the fake anisotropic stylized function a bit further, I learn a trick about hair tangent to make it look like real anisotropic shader.

Then I add in a l detail normal map blended with the hair tangent normal so that the hair surface will have both anisotropic look and subtle details from the normal map.

Pisciasa Hair Demo:

Tessellation trick for smoothing the model surface

I turned on PN_triangle tessellation for most of my shader since they are safe to use on a static mesh, they give me nice smoother surface model. I really like this tessellation effect since they manage to keep the model original sharp edge and just add a subtle smoothness to the surface curvature.

Look dev Tip & tricks

Because my shaders and graphs involve in many nodes and function, apart from using comment group or reroute pins, I have a number of tips & tricks to organize and speed things up.

Using Parameter where possible

I always build my graph directly inside my main testing shaders with the use of parameter everywhere if possible since that would save me a lot of time tweaking numbers and values and then wait for the shader to compile again. Using parameters might increase the overhead compiling time a little bit at the beginning but the real-time update features worth it.

Clean up and keep things organized as I build it

Surprisingly, cleaning up and keeping things organized as I build the graph does not slow me down, instead it boost up the whole process greatly, because everything are well organized and documented and so it is easy to grasp the structure, I can quickly troubleshoot whenever a bug happen or need to change/improve any single module in the graph.

Optimize later

Once I am happy with the whole graphs then I will start to optimize the graph by converting unused parameter into static, revise the graph to see if I can simplify the flow by using fewer nodes or using more optimal flows.

Packaging for reuse

I spent a lot of time to test my graph with different models and params to make sure it’s a decent function that I can re-use over time. Once I am happy with the result, I pack the whole graph and convert it into a material function with proper input and output channels. Replace the whole complex graphs in the main shader with a compact material function to make things much more readable and a lot easier to maintain and update whenever I have newer/better version of the function. I have been working on my own UE4 material functions & shaders library over time, here is a quick look at what I am having in hand, part of this library is included in Pisciasa Project.

Lighting

This is the view of the whole scene of Pisciasa lighting setup, I used over 50 lights in UE4 to light the final scene, however, thought process doesn’t seem that complicated. I started with key light first to lock down the very key lighting like in the concept, then I add fill light + tweaking the skylight/environment to match the shadow area color,

Then lastly I move onto to adding tiny backlight along with local lights all over the place (there are 38 of them!) to try to re-create the same feeling of color and lighting effect as in the concept.

Volumetric light

Before doing the concept of Pisciasa I have played around with UE4 volumetric lighting in its new update of version 4.18 and I really like the quality of it so I am quite confident to put these effect into the final render, before that I was intended to use alpha cards to fake the effect but now I can get real volumetric lighting with shadow within UE4 which really boost the mood up the way I want.

Render for quality

My workflow for rendering quality images in UE4 is to capture the scene at a really high resolution like 100 Megapixel images (10k x 10k image) then down res it into a much smaller image to get super nice Anti Alias and detail render quality.

In all of my previous projects, I used UE4 native tool to capture high-resolution screencap which is not really stable and tend to freeze UE4 and my 1080ti when rendering large resolution. So this is where Nvidia Ansel come in.

Ansel is original a program developed by Nvidia to capture hi-res screenshot in PC games, Now it comes with UE4 as a plugin which you can run during “Play In New Window“ mode.

It did take me awhile to get used to using Ansel after a series of fails and trials before I could come up with a workflow that can Ansel. Here are some of the key notable points:

Use Level Sequencer and make it autoplay when the level starts.
Import your FBX camera into the sequencer, mark it as possessive so that it can retain its non-keyable value during play mode.
Use Camera Track to control which camera you want to capture
Play in New Window mode must have the same resolution ratio as your camera sensor width & height ratio, otherwise there will be black banding in you hires screencap.
Use Enhance checkbox of Ansel to get better DOF effect.

Summary and future plan

Everything above is pretty much all of my experiences and techniques I have gained so far while working on Pisciasa. There are still a lot of uncharted water areas here and there that I definitely need to work on like stylized shader that really reacts to real light in the scene or finding a way to get better shadow rendition and control, etc. However on top of that, one critical thing that I have learned is time management and better planning, without these two I wouldn’t be able to finish on time with this much ambition for Pisciasa. I now know what I can achieve over the course of 3 months of my free time. It will be much easier for me and I am feeling a bit more confidence when planning ahead any future project with much better quality and bigger ambition from what I have learned. Thank you for your time reading this article and hope you enjoy it and stay tuned for my next projects in the upcoming time! PDD out.

Pisciasa Turntable:

Interested in taking a deeper look at Pisciasa? Get the packs here and there and see it for yourself!

Dzung Phung Dinh, Founder – Thunder Cloud Studio

Dzung Phung Dinh

Dzung Phung Dinh, a leading veteran 3D artist with incredible passion and decades of valuable experience showcasing his exceptional talent and expertise in the CG industry. Dzung’s career includes contributions to major games and Hollywood visual effects, working with industry giants like Rare-Microsoft Studio and Double Negative, being the Art Director of The Color Club. His notable prjeocts are Skyfall 007, Snow White and the Huntsman, John Carter of Mars and various AAA games for Xbox 360. He has been featured in prominent 3D magazines with big achievements such as winning first prize at CGSA (2010), the Outstanding Production Award at Blue Zone (2010), creating the award-nominated VR film Painted Flower (2016), and serving as the Art Director of Subverse (2019). In 2013, Dzung founded Thunder Cloud Studio with the core value being the highest quality as himself – an elite artist without any quality compromise.