One of the most amazing things for me as an artist that current generation brought with it was the introduction of the concept of shaders. No longer did we describe surfaces merely by flat images that already incorporated the majority of the lighting. Our materials became vastly richer with much more dynamic per-pixel lighting as well, and hands down had the biggest impact on the visual definition of the current generation of real-time graphics.
But the programmable pixel and vertex pipelines were capable of much more than just calculating normal map and specular contributions. Also that wide array of things you could do with a pixel was now at the fingertips of a much broader audience, including the visual content creators themselves, who finally got a chance to become the architects of their own tech. From animated wet surfaces in a Modern Warfare ship level to heightmap-based vertex and up-aligned blending popularized by Uncharted 2, shaders further drove the visual splendor that is this generation of games.
But such advancements came at a price:
Technological: the amount of textures needed to define a single surface grew exponentially, as well as the amount of video memory needed for their allocation.
Productional: The amount and sophistication of content skyrocketed. Budgets are through the roof in this generation of games and team sizes in some cases are plain ridiculous coming close to a thousand people. Of course it's not completely the fault of shaders, but art assets production is still one of the most expensive parts of your budget.
Now as an industry and as a consumerist society, I guess, we are always about more stuff. Number 1 thing your average Joe Gamer seems to want from every new game is better graphics. Most people naturally see it as more polygons and crisper textures, since they seem like things that drive the visual quality of games. And they certainly do, but unfortunately they have a pretty remote impact on the visual pleasure that players experience with our games.
When I come back to the old games that I used to enjoy 10 or even 5 years ago I'm amazed at just how crude the technology was back then: the resolutions, the polycounts, the effects. Yet somehow I and millions of people who played that game as well, managed to enjoy it on all levels. Including the visual.
How so? Some might say that people are always dazzled by the top technology of their time. But I reckon you this: People are dazzled by Beauty.
As an artist I've always been fascinated and interested in the concept of Beauty and I study it always with everything I do. I've been kindly invited by the top universities and art academies of the country, to share my findings on that particular subject, so you could say some of my ideas about Beauty were “ok”. Now in this paper I'm not going to discuss what Beauty is, but I can definitely tell you what it isn't. Beauty never lies in the details.
I so love this image, since it illustrates the point so vividly. It is a quick sketch that has very little in terms of detail, yet it makes it so painstakingly obvious why it is a sight worth seeing. The amazing lights and colors take your breath away, completely disregarding the fact there isn't a remotely detailed object in the whole image.
The visual world around us is infinitely complex. All visual representative art is merely an approximation, comprised with the limited resources we have at our disposal. We can never have enough resources to recreate every single process that shapes the world around us, so trying to distill it to what makes reality feel real and feel beautiful is key in balancing the quality of your art with the amount time required to produce it. Another amazing example would be Robh Ruppel with the way he breaks down images to simple shapes and gradients to eventually come out with an almost photorealistic image that actually reeks with “simplicity” when you just take a closer look.
Now with this logic in mind and a little fetish for smart tech I was always fascinated with the concept of procedural materials. Also, in my own work I kept noticing little things here and there, like people ignoring that wood, plastic and metal could have the same base diffuse texture as long as you sell their specular properties or wear and tear correctly. Or that my tileable textures diffuse could at times be just a kind of surface noise if it wasn't for the AO on top.
Somewhere along that time I stumbled upon another interesting piece of shader tech: Gradient Mapping. All it does is take a grayscale heightmap you feed it and paint it with colors you assign to different pixel heights (brightness values).
For example, here all pixels with brightness from 0.0 to 0.33 will gradually transition from red to green, .33-.66 will transition from green to blue and blue eventually to yellow. You've seen that technology used in Left for Dead 2 by Valve, where it allows them to pack numbers of blood splatter and detail onto a single zombie texture, as well as to variably color them at runtime.
So with another personal project I set out to do I really wanted to push the concept and see how far you can go with “procedural” materials even on current-day tech. After months of hard work, here it is (be sure to HD):
Not a single surface here uses a dedicated RGB diffuse texture. In fact I wanted to take it as far as inputting only one texture per one type of surface. Now of course there were some other little textures buried in the shader functions but they were no more than small grayscale masks stacked together. But let's take it from the top. The main idea in this procedural approach is this:
Separating Surface Volume from Surface Detail. As simple as this. Generally rocks in the same area will have a similar geological origin thus requiring a similar diffuse rock texture component. All objects in the same environment usually accumulate identical types of dirt. Objects made from the same materials will generally wear, tear and decay similarly. Objects in a damp environment will start growing a similar type of moss, etc.,etc. What differs is how this dirt, moss, wear, tear accumulate on the surface, as well as the diffuse texture pattern.
Now it may look like a lot of textures, but don't forget that all those grayscale ones are always stacked into R, G and B channels of a single DXT1 texture and are reused everywhere. Let's take a look at the memory usage in the worst case scenario, when we just have 2 materials to compare:
Now whether we import our heightmaps as an alpha channel in a DXT5 texture or a separate grayscale texture we’ll still end up using another 128kb for 512x map. We can stack 3 different heightmaps into a single DXT1 texture and save 1/3 of 128kb but there is a better way.
A normal map's blue channel hardly stores any vital information, so why don’t we put it to meaningful use? You don’t even need to recreate the blue channel in-shader – merely replacing it with a neutral normal color works fine in most situations. All and all you’ll spend 2 additional pixel shader instructions, which is a puny price to pay for cutting down the memory footprint by half.
And this is exactly why further on you’re going to see Normal Maps and Height Maps listed as one single entity.
Now imagine you wanted to vertex blend damage to your material as a lot of games do this day. Here's how it goes:
With this tech you get your damage smartly blended only where it could exist in real life – on the most protruding parts of your surface volume. Now on top of getting a 2 times memory gain you get an opportunity to dynamically tile, tweak intensity and available range of your damage, which by the way will always be taken into account in every kind of heightmap-based blending further on. And you can change it per every single material instance, creating exactly the type of damage you need.
Now imagine that we, as it usually happens, need a bunch of similar surfaces for our level:
Almost 2 times difference. Want to count what the difference would be if we also did a damage pass for each of those textures? I took that liberty and did that for you: 488kb to 1576kb. Now how do you like that?
A reasonable question now would be just how much different surfaces could be considered similar and produce acceptable results if we just keep swapping the combo normal and height map. Turns out: almost all of them.
How is that possible? In the beginning we said that the surface volume and diffuse texture patterns were an integral part of any material. But the big thing is, they do not have to be separate entities. You don't have to strictly feed heightmaps to gradient mapping. In fact feel free to forget the term heightmap for now, because from now on it'll be a part of a broader term:
Gradient Map is your main diffuse component so you make it work as such. Blend your depth info with AO and every single texture pattern you might need. Paint in all the details and accents or tweak surface values just like you would with a regular texture. If you're working with heavily photo sourced textures, it's even easier because you can generate a normal map and a heightmap from your diffuse. And then you blend your grayscale diffuse with your heightmap to create a Gradient Map, thus keeping both your surface depth and surface detail info. If you think about surface detail, it is still depth just on a much smaller scale, so Gradient Mapping function processes it greatly.
Think of it as evaporating water from juice or soup to create a concentrate. Colors are water that we can spruce back in at runtime.
The Gradient Map works as good as the Height Map for blending, so no worries here. Also an amazing unexpected use is as an Opacity Mask, for foliage for example. You just have to make sure that your background is completely black and your gradient has its level pushed up so no pixel there is black also. Then you just clip this opacity mask from zero brightness and that's it.
Another amazing fact is that gradient mapping works perfectly well even if you use just your usual diffuse turned grayscale. Here's a comparison of a material from Epic and the same material only with the diffuse map desaturated and put through the GM function.
Now does this difference really warrant a whole diffuse texture for this and every similar object?
The silver lining here is that you can actually use your diffuse textures as Gradient Maps without any modification, right here right now, significantly trimming your texture footprint. Extremely comfy and easy for new tech penetration.
Now's the time for a little pros and cons, which should cover the questions I'd be having right now if I were you:
- Procedural Colors in Tileable Textures vs. Procedural Colors in Uniquely Mapped Textures
The point that I was most worried about as an artist was color. How much do we need? To my surprise 4 colors to color a Gradient Map are more than enough! I even had to create a lighter version of the Gradient Mapping function that just blends between two colors and has no texture overlay. Now I've obviously used it mostly for environment textures, which are generally tiled thus requiring certain uniformity from their colors to make tiling seem unapparent, and that actually works greatly to the advantage of gradient mapping. If you're working with tiled textures a lot – you definitely want to try this out. And if you're not working with tiling textures a lot, how the hell do your games even work?
Also important to note, from an artistic standpoint, good lighting, fog and post processing greatly influence colors usually creating the broadest and most important strokes. Your scene hardly ever is supposed to be about every little piece screaming for attention with a different color. Uniformity is good in a lot of ways and it is definitely not something to be fighting with a lot of the time.
Now you don't have to take my word for what the textures should look like. To make it fair, let's analyze a real-life example and see what makes a high-quality modern day texture:
These textures are made in the good old RGB-diffuse-map fashion, yet you can see that all that grunge and damage is really just an additional layer on top of the base textures. And if you strip that away, the textures have a of lot shades of pretty similar colors – it's the overall color tone and the brightness of each pixel that describes them. I hope that previous examples have convinced you that “Procedural” materials could do all of that – red brick/white bricks, dirty/clean, damaged – in a matter of a few button clicks, saving you a lot of production time and memory as a bonus.
By now you probably think “Ok, what about uniquely mapped stuff?!”. Do not despair. Valve has used it to create vast varieties of undead hordes for Left for Dead 2 and so can you.
You'll hardly want to Gradient Map your main characters but there's still a lot of mileage you can get out of it. Read all about it here
The main hurdle is of course: color variety. Whenever you need a lot of color in a single texture Gradient Mapping probably won't suit your needs. But that's a good thing - you have a choice of what technology to apply to get the biggest bang for your buck. Gradient Mapping and Diffuse Mapping were never meant to be mutually exclusive.
- Relinquishing control vs. Unexpected Variety:
Another tricky thing for me as an artist was fear of losing complete and utter control. Now I'm the first guy to be plain anal about every little piece of my artwork, I'll be carefully planting old pieces of chewing gum and cigarette buds on my textures in places where most people won't even bother. Yet throughout my career I constantly had to teach myself to choose my battles wisely. I don't believe in things existing for their own sake. Anything is only as good as it performs its purpose – no more, no less. And the purpose of details is to be sufficient enough not to break the illusion of the imposed reality, as much as beginning artists think detail is all there is. I was and probably still am a guy who loves his details, yet I had to admit that most players will never notice a difference between Gradient Mapping and Diffuse mapping, just like all the professional artists who I've shown this environment to.
Only further down the road I noticed just how much flexibility I could get from this system. From plants to rocks to bark it's a matter of swapping the only texture and tweaking the colors, diffuse patterns, damage and specular values. One second you use this texture as an orange canyon wall, next one it's already blue and is a part of a cave. Your bark is brown, but just turn it green, tile it more and there you have vines texture. Need more cold color in your shadows? Just make them appear in the cavities of your surfaces to amplify the effect. Want it dirty – just push a button and determine how much. Mossy/sandy - all just a matter of a button push. Procedural damage was another unexpectedly awesome thing since I could reuse a single asset with different damage tiling and intensity and create a whole lot more variety then I could ever imagine doing it the old way. And then I could paint my cracks green and invert their intensity, to create an illusion of vines overgrowing assets further in the distance.
As an artist I've got to grips with this workflow. I can't imagine not being able to tweak any object's color, diffuse noise scale or dirt at my smallest whim. I love it now. It's like working with bigger Lego pieces. We no longer model every part of our level individually, but rather create a set of modular meshes to work with. Then why on earth shouldn't we do it with our materials and textures?!
- Extra Processing Power vs. Freed up VRam
While “procedural” texturing technology frees up a whole lot of VRam it also requires additional processing power. And as always with software optimization - it's a question of what you've got to spare.
If we look at something like PS3 we'll find just 256mb of VRam and it shows. Even the most gorgeous games sometimes put blurry textures right in your face. Notice the dramatic texeleration difference in characters and the background truck in the screenshot above. Yet PS3 has 8 SPUs that technically could be used to alleviate the issue by implementing Gradient Mapping Shaders, cutting the diffuse texture footprint in half or even by 3/4, allowing to selectively increase texeleration on some surfaces. Xbox 360 has 2 SPUs and a combined Ram/VRam module yet it's still half the amount of memory of an iPad. With the Desert environment I didn't use diffuse textures at all, just a couple of masks that are insignificant in terms of the whole level's memory footprint. So it would be fairly accurate to say that I cut my texture memory expenses in half by relying on this technology. It's also worth noting that to create damage I use in shader normal map generation from a grayscale mask which is somewhat of a pricey operation (though there could be plenty of workarounds). Yet on a modern-day PC the environment you've seen has no trouble doing 60+ FPS. If we can trust UDK's custom nodes instructions calculations then it would take us just 14 instructions to replace a diffuse map with a gradient map. I have fully functional shaders that with 73 instructions provide you full diffuse, specular, masked opacity and normal functionality with only One texture sampled. The most complex version of the shader is 153 instructions and features vertex paintable heightmap-based sand blending as well as 2 types of procedural “smart” damage generation that are also both vertex paintable. These instruction counts are at the very least comparable to the instruction counts of Unreal Engine Games Materials with similar functionality.
Yet I’m not a graphics programmer by any means and UDK's instruction counter does lie from time to time. Right now some proper tests are being conducted and so far I’m afraid I can’t tell you much more. Though I promise to update this with more info the second it appears.
- Production Costs
This one has no cons. Procedural Materials save a lot of production time and subsequently a ton of money. If I had to create every single diffuse map by hand I can assure you that the environment you've seen would've been much smaller or taken more time to produce.
It's funny how explaining this to a more business savvy person the first question I got was: “So how many people can you replace?”. And this is definitely not about replacing people. It's about how much more and how much faster you can produce. There is always lack of time in our industry and having a chance to free some of it up for more important things is an amazing opportunity to have that yields better games and subsequently more profit.
Now there's no question left for me, the notion of diffuse textures being indispensable the way they are, couldn't be further from the truth. Gradient Map carries enough information to make your brain perceive surfaces as completely believable. And that's all there is to it.
So if there's somewhere we can trim fat, it's in the diffuse textures. Both in terms workload and technical constraints. Without sacrificing visual quality.
Now I know our industry is all about more stuff. The next generation is just around the corner and it will have to blow something like Battlefield 3 out of the water if Sony and Microsoft are going to convince their audience to make the pricey upgrade. As much as I look forward to quadrupling texture sizes and polygon budgets, the industry could well collapse under its own weight. Nowadays, in the world of big games, good-yet-not great-70%-metacritic games just do not pay off any more, and companies are uber reluctant to invest money in something that, in their eyes, doesn't absolutely guarantee return. Now consider investing in big next gen projects, which could very well be double the cost of today's AAA game, for a platform that has zero market penetration? When can you really expect at least 3 million units shipped there, to at least make it worth your while?
The risks could just keep rising proportionally with graphical fidelity. Now there is a cushion in the form of assets being downsized for today's games so going higher-rez could merely be a matter of not resizing, but higher resolutions are still going to require more work.
If we as industry want to stay competitive we should not only advance the quality of the product we produce, but the quality of production itself as well. The industry should slowly move towards things becoming more “procedural”. In fact it does: we switched from animating dudes being blown away by a shotgun to simulating it; we have specific tools to build trees, roads, terrain, or LoDs; we no longer model every single piece of our levels, but rather create a set of highly modular Lego pieces to build our levels with; gameplay scripting in UDK is now visual node-based editing – no more writing code! All of this saved our industry years of man-hours and millions of dollars in production costs. And I believe that this is something we should be doing with materials. Give our artists bigger Lego pieces so they can dedicate themselves to the bigger picture with no real loss in detail. I would love to see next-gen engine creators take that into account.
Normal + Gradient map RGB combo, built-in gradient mapping functionality or even high-level procedural material editor with built in variety of diffuse patterns, specular presets, dirt, damage and wear types where users just input their Gradient Map, tweak a few handles and: voila! Procedural wet effect or water rolling down oblique surfaces, overgrown with moss, dusty, sandy, burned; procedural polished metal, car paint, glass, water etc. – all with one button push. And with deferred lighting becoming commonplace imagine using Screen Space AO to mask tiled dirt to make it appear only in cavities, corners and intersections of flagged static objects. Or use vertex normals to procedurally create a mask for sharp corners that you multiply with a wear map and use it to reveal metal underneath worn painted surfaces, or even use pixel normals from you scene's Normal Pass, combine it with highly contrasted depth pass to try to figure out where those sharp edges are.
There are millions of ideas and some of them could save you months of work and hundreds megabytes of VRam. Technology is meant to be a tool to achieve artistic results, so instead making us try to keep up with the crazy amount of fidelity we have to put in, lets make it help us concentrate on adding details and creating real beauty where it would matter most.
Thank you very much and keep it pretty.
Andrew Maximov, 2012
P.s. There one last thing I wanted to share with you guys...but I can't seem to remember what it is....
...oh yeah, THE MATERIALS!
Click here to grab them! :)
These are all kinds of "procedural" materials for you to check out as well as a couple of example textures and meshes. Also there's a .PSD that makes for very smooth gradient map production as it allows you to preview your gradient mapping, normal, specular, damage and diffuse pattern influence right in Photoshop! It's like you're painting your gradient map directly in UDK and can immediately see the end result! I've made a little video that will hopefully make things more visual for you:
But the programmable pixel and vertex pipelines were capable of much more than just calculating normal map and specular contributions. Also that wide array of things you could do with a pixel was now at the fingertips of a much broader audience, including the visual content creators themselves, who finally got a chance to become the architects of their own tech. From animated wet surfaces in a Modern Warfare ship level to heightmap-based vertex and up-aligned blending popularized by Uncharted 2, shaders further drove the visual splendor that is this generation of games.
But such advancements came at a price:
Technological: the amount of textures needed to define a single surface grew exponentially, as well as the amount of video memory needed for their allocation.
Productional: The amount and sophistication of content skyrocketed. Budgets are through the roof in this generation of games and team sizes in some cases are plain ridiculous coming close to a thousand people. Of course it's not completely the fault of shaders, but art assets production is still one of the most expensive parts of your budget.
Now as an industry and as a consumerist society, I guess, we are always about more stuff. Number 1 thing your average Joe Gamer seems to want from every new game is better graphics. Most people naturally see it as more polygons and crisper textures, since they seem like things that drive the visual quality of games. And they certainly do, but unfortunately they have a pretty remote impact on the visual pleasure that players experience with our games.
When I come back to the old games that I used to enjoy 10 or even 5 years ago I'm amazed at just how crude the technology was back then: the resolutions, the polycounts, the effects. Yet somehow I and millions of people who played that game as well, managed to enjoy it on all levels. Including the visual.
How so? Some might say that people are always dazzled by the top technology of their time. But I reckon you this: People are dazzled by Beauty.
As an artist I've always been fascinated and interested in the concept of Beauty and I study it always with everything I do. I've been kindly invited by the top universities and art academies of the country, to share my findings on that particular subject, so you could say some of my ideas about Beauty were “ok”. Now in this paper I'm not going to discuss what Beauty is, but I can definitely tell you what it isn't. Beauty never lies in the details.
Still frame from my “Analyzing Beauty” Lecture, image by Zhu Haibo
I so love this image, since it illustrates the point so vividly. It is a quick sketch that has very little in terms of detail, yet it makes it so painstakingly obvious why it is a sight worth seeing. The amazing lights and colors take your breath away, completely disregarding the fact there isn't a remotely detailed object in the whole image.
The visual world around us is infinitely complex. All visual representative art is merely an approximation, comprised with the limited resources we have at our disposal. We can never have enough resources to recreate every single process that shapes the world around us, so trying to distill it to what makes reality feel real and feel beautiful is key in balancing the quality of your art with the amount time required to produce it. Another amazing example would be Robh Ruppel with the way he breaks down images to simple shapes and gradients to eventually come out with an almost photorealistic image that actually reeks with “simplicity” when you just take a closer look.
Painting by Robh Ruppel
Now with this logic in mind and a little fetish for smart tech I was always fascinated with the concept of procedural materials. Also, in my own work I kept noticing little things here and there, like people ignoring that wood, plastic and metal could have the same base diffuse texture as long as you sell their specular properties or wear and tear correctly. Or that my tileable textures diffuse could at times be just a kind of surface noise if it wasn't for the AO on top.
Somewhere along that time I stumbled upon another interesting piece of shader tech: Gradient Mapping. All it does is take a grayscale heightmap you feed it and paint it with colors you assign to different pixel heights (brightness values).
For example, here all pixels with brightness from 0.0 to 0.33 will gradually transition from red to green, .33-.66 will transition from green to blue and blue eventually to yellow. You've seen that technology used in Left for Dead 2 by Valve, where it allows them to pack numbers of blood splatter and detail onto a single zombie texture, as well as to variably color them at runtime.
Screenshot from Valves “Left for Dead 2”
Procedural Environment
So with another personal project I set out to do I really wanted to push the concept and see how far you can go with “procedural” materials even on current-day tech. After months of hard work, here it is (be sure to HD):
Not a single surface here uses a dedicated RGB diffuse texture. In fact I wanted to take it as far as inputting only one texture per one type of surface. Now of course there were some other little textures buried in the shader functions but they were no more than small grayscale masks stacked together. But let's take it from the top. The main idea in this procedural approach is this:
Separating Surface Volume from Surface Detail. As simple as this. Generally rocks in the same area will have a similar geological origin thus requiring a similar diffuse rock texture component. All objects in the same environment usually accumulate identical types of dirt. Objects made from the same materials will generally wear, tear and decay similarly. Objects in a damp environment will start growing a similar type of moss, etc.,etc. What differs is how this dirt, moss, wear, tear accumulate on the surface, as well as the diffuse texture pattern.
Now it may look like a lot of textures, but don't forget that all those grayscale ones are always stacked into R, G and B channels of a single DXT1 texture and are reused everywhere. Let's take a look at the memory usage in the worst case scenario, when we just have 2 materials to compare:
Optimizing Storage of Heightmaps
Now whether we import our heightmaps as an alpha channel in a DXT5 texture or a separate grayscale texture we’ll still end up using another 128kb for 512x map. We can stack 3 different heightmaps into a single DXT1 texture and save 1/3 of 128kb but there is a better way.
A normal map's blue channel hardly stores any vital information, so why don’t we put it to meaningful use? You don’t even need to recreate the blue channel in-shader – merely replacing it with a neutral normal color works fine in most situations. All and all you’ll spend 2 additional pixel shader instructions, which is a puny price to pay for cutting down the memory footprint by half.
And this is exactly why further on you’re going to see Normal Maps and Height Maps listed as one single entity.
Damage
Now imagine you wanted to vertex blend damage to your material as a lot of games do this day. Here's how it goes:
With this tech you get your damage smartly blended only where it could exist in real life – on the most protruding parts of your surface volume. Now on top of getting a 2 times memory gain you get an opportunity to dynamically tile, tweak intensity and available range of your damage, which by the way will always be taken into account in every kind of heightmap-based blending further on. And you can change it per every single material instance, creating exactly the type of damage you need.
Multitude
Now imagine that we, as it usually happens, need a bunch of similar surfaces for our level:
Almost 2 times difference. Want to count what the difference would be if we also did a damage pass for each of those textures? I took that liberty and did that for you: 488kb to 1576kb. Now how do you like that?
A reasonable question now would be just how much different surfaces could be considered similar and produce acceptable results if we just keep swapping the combo normal and height map. Turns out: almost all of them.
How is that possible? In the beginning we said that the surface volume and diffuse texture patterns were an integral part of any material. But the big thing is, they do not have to be separate entities. You don't have to strictly feed heightmaps to gradient mapping. In fact feel free to forget the term heightmap for now, because from now on it'll be a part of a broader term:
Gradient Map
Gradient Map is your main diffuse component so you make it work as such. Blend your depth info with AO and every single texture pattern you might need. Paint in all the details and accents or tweak surface values just like you would with a regular texture. If you're working with heavily photo sourced textures, it's even easier because you can generate a normal map and a heightmap from your diffuse. And then you blend your grayscale diffuse with your heightmap to create a Gradient Map, thus keeping both your surface depth and surface detail info. If you think about surface detail, it is still depth just on a much smaller scale, so Gradient Mapping function processes it greatly.
Think of it as evaporating water from juice or soup to create a concentrate. Colors are water that we can spruce back in at runtime.
The Gradient Map works as good as the Height Map for blending, so no worries here. Also an amazing unexpected use is as an Opacity Mask, for foliage for example. You just have to make sure that your background is completely black and your gradient has its level pushed up so no pixel there is black also. Then you just clip this opacity mask from zero brightness and that's it.
Another amazing fact is that gradient mapping works perfectly well even if you use just your usual diffuse turned grayscale. Here's a comparison of a material from Epic and the same material only with the diffuse map desaturated and put through the GM function.
Now does this difference really warrant a whole diffuse texture for this and every similar object?
The silver lining here is that you can actually use your diffuse textures as Gradient Maps without any modification, right here right now, significantly trimming your texture footprint. Extremely comfy and easy for new tech penetration.
Now's the time for a little pros and cons, which should cover the questions I'd be having right now if I were you:
- Procedural Colors in Tileable Textures vs. Procedural Colors in Uniquely Mapped Textures
The point that I was most worried about as an artist was color. How much do we need? To my surprise 4 colors to color a Gradient Map are more than enough! I even had to create a lighter version of the Gradient Mapping function that just blends between two colors and has no texture overlay. Now I've obviously used it mostly for environment textures, which are generally tiled thus requiring certain uniformity from their colors to make tiling seem unapparent, and that actually works greatly to the advantage of gradient mapping. If you're working with tiled textures a lot – you definitely want to try this out. And if you're not working with tiling textures a lot, how the hell do your games even work?
Also important to note, from an artistic standpoint, good lighting, fog and post processing greatly influence colors usually creating the broadest and most important strokes. Your scene hardly ever is supposed to be about every little piece screaming for attention with a different color. Uniformity is good in a lot of ways and it is definitely not something to be fighting with a lot of the time.
Now you don't have to take my word for what the textures should look like. To make it fair, let's analyze a real-life example and see what makes a high-quality modern day texture:
Textures from Uncharted 3 from Naughty Dog by Melissa Altobello
These textures are made in the good old RGB-diffuse-map fashion, yet you can see that all that grunge and damage is really just an additional layer on top of the base textures. And if you strip that away, the textures have a of lot shades of pretty similar colors – it's the overall color tone and the brightness of each pixel that describes them. I hope that previous examples have convinced you that “Procedural” materials could do all of that – red brick/white bricks, dirty/clean, damaged – in a matter of a few button clicks, saving you a lot of production time and memory as a bonus.
By now you probably think “Ok, what about uniquely mapped stuff?!”. Do not despair. Valve has used it to create vast varieties of undead hordes for Left for Dead 2 and so can you.
Viewport screenshot of a “Left For Dead 2” character by Valve
You'll hardly want to Gradient Map your main characters but there's still a lot of mileage you can get out of it. Read all about it here
The main hurdle is of course: color variety. Whenever you need a lot of color in a single texture Gradient Mapping probably won't suit your needs. But that's a good thing - you have a choice of what technology to apply to get the biggest bang for your buck. Gradient Mapping and Diffuse Mapping were never meant to be mutually exclusive.
- Relinquishing control vs. Unexpected Variety:
Another tricky thing for me as an artist was fear of losing complete and utter control. Now I'm the first guy to be plain anal about every little piece of my artwork, I'll be carefully planting old pieces of chewing gum and cigarette buds on my textures in places where most people won't even bother. Yet throughout my career I constantly had to teach myself to choose my battles wisely. I don't believe in things existing for their own sake. Anything is only as good as it performs its purpose – no more, no less. And the purpose of details is to be sufficient enough not to break the illusion of the imposed reality, as much as beginning artists think detail is all there is. I was and probably still am a guy who loves his details, yet I had to admit that most players will never notice a difference between Gradient Mapping and Diffuse mapping, just like all the professional artists who I've shown this environment to.
Only further down the road I noticed just how much flexibility I could get from this system. From plants to rocks to bark it's a matter of swapping the only texture and tweaking the colors, diffuse patterns, damage and specular values. One second you use this texture as an orange canyon wall, next one it's already blue and is a part of a cave. Your bark is brown, but just turn it green, tile it more and there you have vines texture. Need more cold color in your shadows? Just make them appear in the cavities of your surfaces to amplify the effect. Want it dirty – just push a button and determine how much. Mossy/sandy - all just a matter of a button push. Procedural damage was another unexpectedly awesome thing since I could reuse a single asset with different damage tiling and intensity and create a whole lot more variety then I could ever imagine doing it the old way. And then I could paint my cracks green and invert their intensity, to create an illusion of vines overgrowing assets further in the distance.
As an artist I've got to grips with this workflow. I can't imagine not being able to tweak any object's color, diffuse noise scale or dirt at my smallest whim. I love it now. It's like working with bigger Lego pieces. We no longer model every part of our level individually, but rather create a set of modular meshes to work with. Then why on earth shouldn't we do it with our materials and textures?!
- Extra Processing Power vs. Freed up VRam
While “procedural” texturing technology frees up a whole lot of VRam it also requires additional processing power. And as always with software optimization - it's a question of what you've got to spare.
Screenshot from the “Last of Us” by Naughty Dog
If we look at something like PS3 we'll find just 256mb of VRam and it shows. Even the most gorgeous games sometimes put blurry textures right in your face. Notice the dramatic texeleration difference in characters and the background truck in the screenshot above. Yet PS3 has 8 SPUs that technically could be used to alleviate the issue by implementing Gradient Mapping Shaders, cutting the diffuse texture footprint in half or even by 3/4, allowing to selectively increase texeleration on some surfaces. Xbox 360 has 2 SPUs and a combined Ram/VRam module yet it's still half the amount of memory of an iPad. With the Desert environment I didn't use diffuse textures at all, just a couple of masks that are insignificant in terms of the whole level's memory footprint. So it would be fairly accurate to say that I cut my texture memory expenses in half by relying on this technology. It's also worth noting that to create damage I use in shader normal map generation from a grayscale mask which is somewhat of a pricey operation (though there could be plenty of workarounds). Yet on a modern-day PC the environment you've seen has no trouble doing 60+ FPS. If we can trust UDK's custom nodes instructions calculations then it would take us just 14 instructions to replace a diffuse map with a gradient map. I have fully functional shaders that with 73 instructions provide you full diffuse, specular, masked opacity and normal functionality with only One texture sampled. The most complex version of the shader is 153 instructions and features vertex paintable heightmap-based sand blending as well as 2 types of procedural “smart” damage generation that are also both vertex paintable. These instruction counts are at the very least comparable to the instruction counts of Unreal Engine Games Materials with similar functionality.
Yet I’m not a graphics programmer by any means and UDK's instruction counter does lie from time to time. Right now some proper tests are being conducted and so far I’m afraid I can’t tell you much more. Though I promise to update this with more info the second it appears.
- Production Costs
This one has no cons. Procedural Materials save a lot of production time and subsequently a ton of money. If I had to create every single diffuse map by hand I can assure you that the environment you've seen would've been much smaller or taken more time to produce.
It's funny how explaining this to a more business savvy person the first question I got was: “So how many people can you replace?”. And this is definitely not about replacing people. It's about how much more and how much faster you can produce. There is always lack of time in our industry and having a chance to free some of it up for more important things is an amazing opportunity to have that yields better games and subsequently more profit.
Outro
Now there's no question left for me, the notion of diffuse textures being indispensable the way they are, couldn't be further from the truth. Gradient Map carries enough information to make your brain perceive surfaces as completely believable. And that's all there is to it.
So if there's somewhere we can trim fat, it's in the diffuse textures. Both in terms workload and technical constraints. Without sacrificing visual quality.
Now I know our industry is all about more stuff. The next generation is just around the corner and it will have to blow something like Battlefield 3 out of the water if Sony and Microsoft are going to convince their audience to make the pricey upgrade. As much as I look forward to quadrupling texture sizes and polygon budgets, the industry could well collapse under its own weight. Nowadays, in the world of big games, good-yet-not great-70%-metacritic games just do not pay off any more, and companies are uber reluctant to invest money in something that, in their eyes, doesn't absolutely guarantee return. Now consider investing in big next gen projects, which could very well be double the cost of today's AAA game, for a platform that has zero market penetration? When can you really expect at least 3 million units shipped there, to at least make it worth your while?
The risks could just keep rising proportionally with graphical fidelity. Now there is a cushion in the form of assets being downsized for today's games so going higher-rez could merely be a matter of not resizing, but higher resolutions are still going to require more work.
If we as industry want to stay competitive we should not only advance the quality of the product we produce, but the quality of production itself as well. The industry should slowly move towards things becoming more “procedural”. In fact it does: we switched from animating dudes being blown away by a shotgun to simulating it; we have specific tools to build trees, roads, terrain, or LoDs; we no longer model every single piece of our levels, but rather create a set of highly modular Lego pieces to build our levels with; gameplay scripting in UDK is now visual node-based editing – no more writing code! All of this saved our industry years of man-hours and millions of dollars in production costs. And I believe that this is something we should be doing with materials. Give our artists bigger Lego pieces so they can dedicate themselves to the bigger picture with no real loss in detail. I would love to see next-gen engine creators take that into account.
Normal + Gradient map RGB combo, built-in gradient mapping functionality or even high-level procedural material editor with built in variety of diffuse patterns, specular presets, dirt, damage and wear types where users just input their Gradient Map, tweak a few handles and: voila! Procedural wet effect or water rolling down oblique surfaces, overgrown with moss, dusty, sandy, burned; procedural polished metal, car paint, glass, water etc. – all with one button push. And with deferred lighting becoming commonplace imagine using Screen Space AO to mask tiled dirt to make it appear only in cavities, corners and intersections of flagged static objects. Or use vertex normals to procedurally create a mask for sharp corners that you multiply with a wear map and use it to reveal metal underneath worn painted surfaces, or even use pixel normals from you scene's Normal Pass, combine it with highly contrasted depth pass to try to figure out where those sharp edges are.
There are millions of ideas and some of them could save you months of work and hundreds megabytes of VRam. Technology is meant to be a tool to achieve artistic results, so instead making us try to keep up with the crazy amount of fidelity we have to put in, lets make it help us concentrate on adding details and creating real beauty where it would matter most.
Thank you very much and keep it pretty.
Andrew Maximov, 2012
P.s. There one last thing I wanted to share with you guys...but I can't seem to remember what it is....
...oh yeah, THE MATERIALS!
Click here to grab them! :)
These are all kinds of "procedural" materials for you to check out as well as a couple of example textures and meshes. Also there's a .PSD that makes for very smooth gradient map production as it allows you to preview your gradient mapping, normal, specular, damage and diffuse pattern influence right in Photoshop! It's like you're painting your gradient map directly in UDK and can immediately see the end result! I've made a little video that will hopefully make things more visual for you: