Stylized Texture Tutorial

Introduction
This tutorial was designed with people who lack drawing tablets in mind, in hopes that among those people, the ones wanting to work within a cartoon/stylized art style will find this to be beneficial. If you're looking to cut costs so that you don't have to hire a texture artist in the end, this is a great method for going about stylized texture creation.

If you can memorize this (rather short) process, you can produce hundreds of textures in a matter of hours (like I did!).

This tutorial is divided into steps, with each step having a GIMP variant and a Photoshop variant within. This tutorial assumes at least Photoshop CS4 (though CS3 and CS2 probably work just fine), or GIMP 2.6 (available for free at http://www.gimp.org/downloads/).

The original tutorial posted on the Unity forums can be found here:.

Step 1: Find an Appropriate Base Texture


Regardless of what software we use, we need to find a texture that will act as a decent base texture--basically, we have to assess whether it'll look like crap or not by the time we complete this process. For example, to the right of this text is a very detailed grass texture. This texture in particular is too detailed, because of the sheer number of places in which the colors contrast from each other in terms of luminosity (brightness)--in other words, there's far too many places on the texture where different regions of dark colors touch different regions of lighter colors.

A better alternative would be the second image among the images to the right of this text. While there are still many spots where the colors contrast, this is a more ideal texture in that it doesn't have as much contrasting regions throughout. Note that it is not necessarily a bad idea to use more detailed textures, it's just difficult to deal with textures that have more contrasting regions.



Now, forgive me for using grass as an example, because when using this method to create a stylized texture, grass is actually incredibly difficult to get right without over-doing it, simply because it's hard to find base textures that aren't within the range of detail you want. For this example, we'll be using a small, non-seamless texture with details that will bring out what this method is capable of doing--the bottom image on the right, a simple concrete texture.



Step 2: Initial Filtering
This step is used to strip out some of the more unnecessary details that may become issues later on in the process, but is not necessarily required.

GIMP Instructions
WARNING FOR GIMP USERS: Skip this step if your texture has any transparency applied to it. Not skipping this step while processing a transparent image will result in artifacts appearing around the edges of the non-transparent portions of the image!

Go to Filters > Enhance > Despeckle. You should see a window similar to this (click to enlarge):



Leave all of the default values in place, and press OK. Go on to Step 3.

Photoshop Instructions
Go to Filters > Noise > Despeckle. If any window appears asking you to clarify options for this filter (which it doesn't in CS4--can't speak for CS3 or CS2), just leave them as their default settings. Press OK and go on to Step 3.

Step 3: Detail Retention
This step is used to regain some of the detail accidentally lost as a result of the filtering in Step 2. This isn't totally necessary either, but if you noticed a significant decrease in overall detail after Step 2, you may not want to skip this step.

GIMP Instructions
Go to Filters > Enhance > Sharpen. You should see a window similar to this (click to enlarge):



Set the value to be 40, and click OK. Go on to Step 4.

Photoshop Instructions
Go to Filters > Sharpen > Sharpen. As with Step 2, if any window appears asking you to clarify options for this filter (which it doesn't in CS4--can't speak for CS3 or CS2), just leave them as their default settings. Press OK and go on to Step 4.

Step 4: Paint Filtering
This is the most important step. Regardless of whether or not you skipped the above steps, you'll need to do this one. This step is the "fake" painting filter that essentially emulates the style of hand-painted textures. Whether or not it actually looks like a hand-painted texture depends on the settings tweaked during this step.

GIMP Instructions
Go to Filters > Blur > Selective Gaussian Blur. You should see a window similar to this (click to enlarge):



This is the most important step, so you must know your texture well (as outlined in Step 1) in order to make the decisions necessary for this texture to look good in the end.

The Max Delta setting looks for contrasting regions in the image, hence the name "delta" (delta meaning "change in," as in "change in time" or "change in contrast"). Higher values in this field will result in more blurry textures (reducing the hand-painted look), lower values result in negligible blurring, negating the effect this process attempts to pull off. The Blur Amount setting is essentially just a measure of how blurry you want the regions that don't contrast more than the Max Delta setting allows, giving the appearance of different 'paints' being mixed together to create a color.

For this example, we're going to set the blur amount to 5 and the max delta to 20. Press OK. You're done!

Photoshop Instructions
Go to Filters > Blur > Smart Blur.... You should see a window similar to this (click to enlarge):



This tool works similar yet a bit different than its GIMP counterpart. The Radius setting affects the blurriness ("mixed paint" appearance) of the texture, and should stay somewhere between 10 and 40--anything below 10 and your texture may be too blurry, and anything above 40 and the texture may depart from the hand-painted style we're going for, defeating the entire purpose of this process. The Threshold setting affects how Photoshop sees the different color regions in the texture--that is, lower values tell it to look for smaller differences in color to blur, and higher values tell it to look for larger differences to blur--and it should ideally stay between 30 and 65 (<30, and the blurring will have little effect, and >65 will cause contrasting regions to bleed together and kill the texture's overall quality).

Step 5: Final Filtering (Photoshop Only)
This step is entirely optional, and usually requires you to compare the result of skipping this step and following this step, and deciding which one you like better. This step is Photoshop-only because clever use of the Selective Gaussian Blur settings in GIMP will result in never having to use this (plus, GIMP's Sharpen filter is a lot more potent than it needs to be for this).

Go to Filters > Sharpen > Sharpen. This should harden some of the edges between the different regions of color, and reduce any leftover blurriness from the last step. If you don't like this result, undo it (Ctrl+Z or Command+Z on the Mac--I think) and stick with what you've got.

Conclusion
When it's all said and done, you should have a texture similar to this (click to enlarge):



If you don't like the result, tweak the settings in Step 4 and try again. Note that this process should produce no seams on the texture (if not very negligible seams), which can be fixed using the Offset tools in GIMP/Photoshop in combination with the Smudge tool or a bit of feathered blurring.