Making LOD models

[Tels]

Today I finally found some time to start creating an LOD model for one of our high-poly models.

 

A complete step-by-step guide with screenshots will be shortly here:

 

http://wiki.thedarkmod.com/index.php?title=Creating_LOD_Models

 

but I also post the pictures here so you can comment here better.

 

For my test, I selected the cattails4 model:

 

 

Place a patch behind the model, texture it with textures/sfx/white and select the patch, this will color it uniform brownish. Then make a screenshot and load into gimp:

 

 

Cut out the model, the size does not matter yet:

 

 

Select "Select by similiar colors" and click on the brown, then fill the selection with black:

 

 

Now "Resize the canvas". That means you change the dimensions of the image, but don't scale it! Select as a size something that fits closely, and is either square, or in my case 2:1 or 1:2:

 

 

Continued:

[Tels]

Since we placed the smaller layer inside a bigger image, lets change the layer size to the image size "Layer => To Image Size" and fill the rest of the image with black, too:

 

 

(You'll note I could have made a more tight fit, but I forgot to adjust it better. It is not that important, because although it will waste a little bit of texture resolution, we are making something that the player never sees close up, anyway)

 

Now select the black parts ("Select by Color" f.i.) and create a layer mask ("Layer => "Add mask"):

 

 

You should end up with this display:

 

 

Now scale the image to a power-of-2:

 

 

and save it under the final name as TGA. When you save it, you will be asked to confirm that you need to export the layers, and then click "Export". Select RLE encoding (we have lots of uniform colored pixels) and top-to-bottom, then "Save".

 

Continued:

[Tels]

Create a material for the decal:

 

textures/darkmod/decals/vegetation/cattails_green4_side
{
   qer_editorimage textures/darkmod/decals/vegetation/cattail_flat_side
   surftype15
   description "foliage"

   twosided
   nonsolid

   {
       blend diffusemap
       map textures/darkmod/decals/vegetation/cattail_flat_side
       alphatest 0.5
   }

   // TDM Ambient Method Related
   {
       if (global5 == 1)
       blend add
       map             textures/darkmod/decals/vegetation/cattail_flat_side
       scale           1, 1
       red         global2
       green           global3
       blue            global4
   }
   {
       if (global5 == 2)
       blend add
       program ambientEnvironment.vfp
       vertexParm      0       1, 1, 1, 1      // UV Scales for Diffuse and Bump
       vertexParm      1       1, 1, 1, 1  // (X,Y) UV Scale for specular, Z: ambient reflection scale
       vertexParm      2       global2, global3, global4, 1
       vertexParm      3       0
       fragmentMap     0       cubeMap env/gen1
       fragmentMap     1       _black                                              // Bump
       fragmentMap     2       textures/darkmod/decals/vegetation/cattail_flat_side  // Diffuse
       fragmentMap     3       _black          // Specular
   }
}

 

(Some lines here on the forum might be wrapped, watch out!)

 

The important bits are:

 

* twosided

* alphatest 0.5

 

Continued:

[Tels]

Now back to DR:

 

Create a patch mesh (3x3) and place it inside your model, at the middle, and drag it out so it covers the model dimensions:

 

 

Select the newly made material, and in the surface inspector, press "Fit". You might also have to flip the texture vertically.

 

Now repeat this step from the other side, e.g. 90°,creating the side-texture and a side patch.

 

You will end up with something like this, the two patches, and the original model in the middle:

 

 

In my first test, I also did a "top-side" view, and places this as patch at the bottom, horizontally. The theory was that there is something the player can see if he looks straight from the top.

 

However, in game tests showed that it only looked funky (often z-fighting with the floor), and since the player is far away from the model, anyway, he can't see it from straight top. If you view it high above, it will still look quite good due to the patch being vertical and thus partly visible.

 

Note: In the screenshots above, the patches are not exactly center in the model, this was a mistake of mine and has been corrected after I took the screenshots.

 

Now deselect the original model, and export the two patches as ASE model. Reimport them into DR and you are almost done:

 

 

Here is the side-by-side view in DR:

 

 

Continued:

[Tels]

Here are in-game comparisation shots:

 

 

Close up you see the shadow is missing (is it possible that an alphatest texture casts a shadow?),and it also looks flat. However, the player will never see it from this distance.

 

 

Here are both in different distances, the last distance is approx. 1000 units away. You can see that even closer up it becomes hard to see which is which - and this here is under unforgiving conditions: the ambient is very bright, ground texture is too bright and doesn't hide any defiencies. In real maps, it will be indistinguishable from maybe 700 units on.

 

The only difference spottable is that the new model is somewhat darker in appearance, probably an artefact of the alphatest in combination with aliasing. I am not sure it is worth fixing that, tho.

 

 

Next: performance comparisation:

[greebo]

That's a nice tutorial, good work.

 

(Wiki?)

[Tels]

I placed 22x22 = 484 models in a testmap and did rn both at 1920x1600, 4x AA, and 4x texture anisotropy. My system (AMD dual-core, Geforce, driver 185.18.36) didn't struggle with either map, but the FPS was comparable at about 40..45 FPS. Edit: I did this comparisation after already creating the texture-atlas, so with the first version of the model, the new version would actually been slower than the orginal version. Sorry for the mixup but I don't want to redo the test just to show that 4xxx drawcalls are slower than 1xxx End of edit.

 

(The FPS always fluctuate on my system, so they can only be used a a rough guideline).

 

Here are some screenshots:

 

 

 

And the same with r_showTris 1:

 

 

 

So we save a lot of tris. But why isn't the performance not improved? Lets take a look with r_showPrimitives 1:

 

Org: 1181 draws, 351872 tris
New: 4276 draws,  10082 tris

 

Ahha! We save a lot of tris, but waste over three thousand drawcalls. Why? Explained next:

[Tels]

That's a nice tutorial, good work.

 

(Wiki?)

 

Sorry, I am still making screenshots, also need to correct the ASE model,and finish the posting here. I will of course put everything on the wiki, just wasn't that fast

[Tels]

Ok, so why do we waste so many more drawcalls? The reason is that we made two textures, and have two surfaces with one of them each. This means each simple model needs two drawcalls, compared to 1 for the org model. Oops!

 

In TDM v1.02 it is not possible to combine models, this means with the org model, you are stuck with wasting one drawcall per model. With the new model, however, it is possible to just take the original patches as prefab, and "stamp" them over the map. Then you can select them all, and use "Create func_static". This would create one huge model with only two surfaces, so instead of wasting 484 drawcalls, you use only 2.

 

However, in TDM v1.02 it is still not possible to combine models, so if you want to place the newly created model, you end up spending 2 drawcalls for each model.

 

In TDM v1.03, it will be possible to combine models, so this is not as much a concern. However, it would be nice to not waste that much drawcalls in either case. Just imagine you have multiple LOD models, and each of them uses 2 drawcalls instead of 1, and this in addition to the original model, which also uses 1 drawcall.

 

So how do we correct this?

 

The first thing we make is an texture atlas (the description here http://en.wikipedia.org/wiki/Texture_atlas sadly isn't that clear without an image

 

Instead of having two textures, we paste the two images next to each other and create only one material definition for them:

 

 

Instead of 256x512 + 256x512 we end up with 512x512 pixel, so we do not need any additional space. However, we only have one file to load, one material definition, etc. Also, we can texture the patches now with the same material, we just need to make sure to adjust the scale to 0.5 x 1.0 and use the "right/left" buttons to shift the texture:

 

 

Don't forget to rexport the ASE model!

 

In game, however, you will find it won't work, it's better, but still used (apparently) two (instead of four) drawcalls. Why?

 

2136 draws, 10084 tris

 

The reason is that we made a two-sided material. This means, whenever D3 encounters a model with such a material shader, it will take all polygons, and double them, then invert their direction. And, strangely enough, it doesn't add them to the surface, but instead creates a new surface. Two surfaces => two drawcalls. Oops. So how do we fix this?

 

First, make the material onesided again by commenting out the "twosided" keyword in the material shader.

 

   // twosided
   nonsolid

 

Then, take each of the two patches in DR, duplicate it, and rotate it by 180° around the z-axis.

 

Note: Make sure to select all patches before exporting again, the easiest is to draw a brush around your patches, and then use "Select inside".

 

In game, make sure that your model is still visible from all sides.

 

And then, using r_showPrimitives 1 we see now:

 

1082 draws, 10084 tris

 

So we saved half the drawcalls, have no more tris (because the doubling is skipped by D3) and on my system I now get 50..54 FPS (Slower systems will definitely benefit more from this than mine)

 

Done!

[Tels]

(Wiki?)

 

I finished the wiki article. Instead of adding the optimization as an afterthought, I weaved it into the article, so that people can get it right on the first try, unlike me

 

http://wiki.thedarkmod.com/index.php?title=Creating_LOD_Models

 

I also commited my work so far to SVN.

 

Now for the boring part, creating all the other models

[nbohr1more]

You have a great mind for details. I can see how the other (early) Doom 3 modders that tried and failed to get LOD to work would have been stumped by these issues.

 

Kudos and Great Job!

 

[Sotha]

What a nice document. Well done!

 

The first thing we make is an texture atlas (the description here http://en.wikipedia....i/Texture_atlas sadly isn't that clear without an image

 

 

Sorry. I'm probably an idiot, but after this part I do not understand anymore.

 

 

Instead of having two textures, we paste the two images next to each other and create only one material definition for them:

 

Instead of 256x512 + 256x512 we end up with 512x512 pixel, so we do not need any additional space. However, we only have one file to load, one material definition, etc. Also, we can texture the patches now with the same material, we just need to make sure to adjust the scale to 0.5 x 1.0 and use the "right/left" buttons to shift the texture:

 

How having a bigger texture reduces drawcalls? The model still consists of two patches perpendicular to each other, each patches now using this bigger texture? And that takes one material, just like the smaller texture requires one material. I just don't get it.

 

Are these the LOD models which are going to be used by the LOD system?

Also, howsabout AI's? Are they gonna get lowpoly versions when they're far away? Current AI's are zillions of polies, having the poly amount reduced when they're far away would probably be useful.

 

EDIT: zillions tiny edits here and there to help convey confused thoughts.

[nbohr1more]

According to the wiki:

 

http://wiki.thedarkmod.com/index.php?title=LOD

 

... the LOD slated for v1.03 wont be applicable for AI.

 

I suppose you could make an AI script that inherits distance checks from the Interleaved Thinking mechanism then re-skins the AI (If I've read my D3W threads correctly, a Skin can define a whole other model...), but the new "skin" would have to be rig-able with AI animations so I'm not sure how much you could pare-down...

 

Or spawn an AI in place of a statue?

 

(just silly talk... I hope no Holy-War comes of this... )

[Baddcog]

@Sotha, I doubt the AI will see lower poly versions.

1-it's alot of work.

2-they would need rigged (alot more work)

 

I think the drawcalls are combined with the combined material because it's on the same entity. Just like the drawcalls were drastically reduced from your 3d house brushes to models. 3 brushes next to each other with same tex is 3 draw calls, but once combined into one object it's only 1. (1 material per entity not 1 material per 3 entities). So with a large tex you have 1 entity, 1 material, not 1 entity and 2 materials.

-----

@Tels.

Great tut. I really didn't think screens of the model from editor would match the model that well, but it seems to work great.

 

I also didn't realize that making the polys instead of twosided would reduce draw calls. I have a few models that can improve there, lately I've just been doing it anyway. Just easier to see what you're modelling anyway.

 

And not only can those LOD tex be used for LOD models but also as background decal filler. A couple decals behind a few models will 'thicken' it alot and be hard to distinguish in alot of cases.

 

Maybe the Wiki also needs a 'decal' link to that page as it's also a great way to make decal textures easily.

[Tels]

What a nice document. Well done!

Sorry. I'm probably an idiot, but after this part I do not understand anymore.

...

How having a bigger texture reduces drawcalls? The model still consists of two patches perpendicular to each other, each patches now using this bigger texture? And that takes one material, just like the smaller texture requires one material. I just don't get it.

 

Hm, okay let me try to explain (maybe I need to draw a picture):

 

* patch #1 used material #1 => one surface, one drawcall

* patch #2 used material #2 => one surface, another drawcall

 

After combing both image parts into one part:

 

* patch #1 used material #1 (the left half of it) => one surface, one drawcall

* patch #2 used material #1 (the right half of it) => same surface, no extra drawcall

 

 

Are these the LOD models which are going to be used by the LOD system?

 

Among others. I made some entities that use the normal and the new model in different LOD stages. Of course, someone need to make all the other LOD models

 

Also, howsabout AI's? Are they gonna get lowpoly versions when they're far away? Current AI's are zillions of polies, having the poly amount reduced when they're far away would probably be useful.

 

AI are used very seldom in a map, so them having 5000 or 2000 polygons doesn't make much of a difference compared to say 1000 cattail models Also, making a low-res version of the AI needs modeling skills, which I don't have. And I am not sure it would work with the weighting/animations.

 

@nbohr1more: A skin can only change the material, not the model. It is possible to teach the AI switch the model based on distance, but as I wrote, I am not sure it is useful, plus we don't have any AI LOD models anyway. I'd say when someone makes the first working one, I do the LOD code for AI, oK?

[nbohr1more]

Heck, I'm just happy about the foliage.

 

(Plus I see a Horse is being worked on... )

[Tels]

@Tels.

Great tut. I really didn't think screens of the model from editor would match the model that well, but it seems to work great.

 

Yeah, I was using it as a last-resort-first-effort-in-a-ditch method, aka a "placeholder until someone can make the real texture", but it worked much better than I even though.

 

Edit: I got the idea of using screenshots from (I think) Sotha who build his skybox with a similiar technique. I think he used ingame screens, tho?

 

I also didn't realize that making the polys instead of twosided would reduce draw calls. I have a few models that can improve there, lately I've just been doing it anyway. Just easier to see what you're modelling anyway.

 

You will laugh, but until I finished the first twosidd version and noticed it still draws twice the drawcalls, it didn't "hit" me either. Technically, all the bits are know to me, but the implications sometimes don't sink in

 

Edit: badcog: Please note that skins can change twosided materials (like translucent) to onesided (like solid), so not every model can just be switched from two to one sided at the modeling stage. An example would be a glass-window. If you make that two-back-to-back surfaces, then a skin could switch it to solid, but you'd still have two solid faces even tho only one would suffice. Just noting (There is another bug in D3 that a skin can switch the two-sidedness of a model after load, but the original skin will dictate whether the sides are duplicated. E.g. an one-sided original skin will never work with a two-sided sin (the second side will be missing), and a two-sided original skin will never work with a one-sided skin (the second side will still be there).

 

v1.03 has code that can work around that, but it is not yet hooked really into SetSkin() - tio fix that bug we probably should do it there.

 

And not only can those LOD tex be used for LOD models but also as background decal filler. A couple decals behind a few models will 'thicken' it alot and be hard to distinguish in alot of cases.

 

Yeah, I should add this to the wiki as an example.

 

Maybe the Wiki also needs a 'decal' link to that page as it's also a great way to make decal textures easily.

 

Added one to:

 

http://wiki.thedarkmod.com/index.php?title=Decals

[Baddcog]

Vert weighting should work fine for LOD AI if the model swaps would work. It would still be the same skeleton, the skeleton is what controls the model.

Whether or not Tels could get the md5 models to swap I can't say.

But am curious, maybe we should do an md5 test just to see if swaps would work.

 

Still, I'm thinking just for any animated objects that might be made in the future (trees that sway in breezy for example), not AI as I said above, tons of work to redo AI, and as Tels said usually there won't be alot of them, especially at a distance. And player is more likely to see them pop than stationary objects as player has to watch them more carefully.

 

-------@ Tels

I figure these have to be created as entities, not models. But this is going to 'bloat' the model editor. And that might be confusing for some authors who haven't dealt with LOD as they'll be able to create cattail, cattail low poly, etc...

 

We need to figure out a good, solid mod-wide naming structure. Do we just want to add _lod1, lod2 to the end of the model names? I take it lod1 would be the highest poly and the lower poly they go the higher number. That way all models with LOD start at lod1 and just continue... as they won't all have the same number of LODs. (my newer plants will have 4 probably, the cattail has 2).

 

Or maybe better the main/high poly model had NO lod extension. So that would be the base model. ie: tree_01, then tree_01_lod1.... (this way creating models the highest detail is default and first in the list and any models that currently exist don't need name changes)

maybe an editor note on highest poly objects : Only to be used with LOD or in scenes that can handle more polys... use sparingly

 

So if an author doesn't want to use LOD for a model they have their choice of any, but editor note is there so they don't haphazardly use 20 high poly models.

[Tels]

-------@ Tels

I figure these have to be created as entities, not models. But this is going to 'bloat' the model editor. And that might be confusing for some authors who haven't dealt with LOD as they'll be able to create cattail, cattail low poly, etc...

 

We need to figure out a good, solid mod-wide naming structure. Do we just want to add _lod1, lod2 to the end of the model names? I take it lod1 would be the highest poly and the lower poly they go the higher number. That way all models with LOD start at lod1 and just continue... as they won't all have the same number of LODs. (my newer plants will have 4 probably, the cattail has 2).

 

Or maybe better the main/high poly model had NO lod extension. So that would be the base model. ie: tree_01, then tree_01_lod1.... (this way creating models the highest detail is default and first in the list and any models that currently exist don't need name changes)

maybe an editor note on highest poly objects : Only to be used with LOD or in scenes that can handle more polys... use sparingly

 

So if an author doesn't want to use LOD for a model they have their choice of any, but editor note is there so they don't haphazardly use 20 high poly models.

 

Currently our models are just named "cattails4.lwo", and I created a "lod" folder, and put the new model under "cattails4_low.ase" there. This means normal mappers don't see the models until they look for them.

 

(Springheel asked that we have only one LOD folder, e.g. not one for each model. However, I am unclear wheter he wanted one for ALLmodels or one per "category". Per category makes more sense, so you find the nature LOD models under nature/lod, and not with all the rest under model/lod).

 

* For static placement, the mapper needs to select the appropr. model.

* However, for anything the player can get close, to you need for dynamic LOD, and forthat, the mapper needs to place an entity (which contains all the proper model references).

 

So we definitely need both.

[Springheel]

(Springheel asked that we have only one LOD folder, e.g. not one for each model. However, I am unclear wheter he wanted one for ALLmodels or one per "category". Per category makes more sense, so you find the nature LOD models under nature/lod, and not with all the rest under model/lod).

 

Actually, I was thinking of the way dds works. You have the regular model folder tree, just under darkmod/models/lod (darkmod/models/lod/furniture, darkmod/models/lod/nature, etc). That would make it slightly faster to see what lod models are available.

[Baddcog]

mmm, seperate folders just seems like a bad idea to me.

 

When I make a mdel now I already have to dig through

 

model....some long ass path to where I'm going

dds... some long ass path to where I'm going

textures... see above

materials... find correct file

def...

skins...

 

Now a whole new directory to place LOD... arghh.

 

Let's just keep it simple and place them right along side the other model.

 

Put LOD# in their name so it's easy to see they are part of the system...

--------

If we add a new directory we'll have more work to do finding/placing them, more work to do adding path names to skins and def files, and if someone just wants better performance and doesn't want the highest poly but figures the next lower detail is fine (which in case of my newer plant would probably be the best to use it's right there in model/plants, not in lod/models/plants where they won't find it or have to open up a completely different hiearchy to find it. (If it even exists because the main model hiearchy won't have nay clues that it DOES exist)

 

Honestly making a model with all those damn tex files is already a pain in the arse. Making LOD's makes it that much harder, let's not make it unnessecarily harder by adding yet another working directory.

 

PLEASE.

[Tels]

Actually, I was thinking of the way dds works. You have the regular model folder tree, just under darkmod/models/lod (darkmod/models/lod/furniture, darkmod/models/lod/nature, etc). That would make it slightly faster to see what lod models are available.

 

We can easily change it now, as we only have maybe 4 LOD models right now. I have no particular preference atm, so your idea can be done from my POV. Although Badcogg has a point with the "exit the current deep hirarchy and go up a lot and then down a lot just to find the first LOD stage". If you have instead:

 

models/darkmod/nature/cattails.low
models/darkmod/nature/shroom.low
...
models/darkmod/nature/lod/cattails_med.ase
models/darkmod/nature/lod/cattails_low.ase
models/darkmod/nature/lod/shroom_med.ase
models/darkmod/nature/lod/shroom_ultra_low.ase

 

you can easily find the LOD models while looking through the nature folder. Otherwise, you'd always need to go up and down again.

 

@Badcogg: All your other points are somewhat moot, because the action of adding a model is very very seldom done (we have added only a handfull models since v1.0), but the action of "selecting a model in DR" is very often done (as is selecting an entity).

 

In all these cases, it needs to be fast.

 

However, I'd like again to stress the point that from v1.03 on people will place much more entities instead of models in their maps. The reason is that a model is basically just a func_static entity, but without any further intelligence. But the pre-made entities all have their LOD setup already, so they actually switch their models. Static placed models never switch, so they are only good if the player can't walk up to them. That will be less often the case than the default where the player an walk to/from something.

 

Anyway, any other team opinions on this? Plus, we probably should split up this discussion into an internal thread, and leave this here for discussing how to make LOD models plus announcements of any made. Can some admin please move the relevant posts?

[Baddcog]

I'll just continue til it splits (didn't pay attention to where it was)

 

I'm still working on models, so they aren't moot points imo. Sure, not adding them quickly, but still I don't want more folders to dig through when I do, it just makes it that much tougher/confusing/easier to mess up and require more troubleshooting...

 

I'm just about at the end of my vegetaion experimenting so hopefully I'll be adding at least 10 of each soon: decals, trees, plants and respective LOD's. I'm even considering LOD's for some older models...

But of course mapping eats up time too.

 

I'd be more inclined to have nature/lod than lod/nature if we DO have to add folders. At least in that case you are right there... But then you have alot of LOD folders so if you have 2 open on your desktop (one working, one SVN) it can be confusing to get the right one. Or if I have trees/lod open and plants/lod...

 

But like you said, we don't have alot of LOD's and may never have alot. So why not just put them right next to their respective models with lod at the end? Certian things probably will never require them (crates, carrots, keys). Certian things may be too hard to add them (AI)

 

Not like having

 

tree_01

tree_01_lod_1

tree_01_lod_2

tree_02

tree_02_lod1

 

is going to make the model browser that much harder to find stuff. You just have a few LOD's here and there to skip through. Most of the time those lists are fairly short anyway due to the folder system in place.

 

My point about placing an lod_1 model instead of the base model would be fine in small places where you don't need LOD anyway but may not want to waste resources on the highest poly model. (Either way if the author wants to use an lod model they could dig through 40 folders to find it. I just don't want to dig through 40 folders everytime I adjust/change/add/improve a model.)

 

This way authors can easily see that Tree_01 has lod models and it's best to use an entity for it. If they can't see that there are LOD's (unless they dig through the LOD folder looking for a match) they won't know it's best to use the entity.

So what do they do? use only the high poly? or do create entity, dig through the list, find there is no tree_01 entity, then go back to create model, dig to trees folder and create tree?

And that'll happen alot if they are specifically trying to use the LOD models. If you don't see them in the model browser with the main model there's no way to know without digging.

[Tels]

I think the idea behind the LOD folder is that we already have a lot of models in some directories, and adding 2..5 LOD stages to each will cause a lot of files in one folder. You can't just say "oh we won't have many", usually such things spiral out of control over time

 

Of course, you are right, it sorts better and can be easily seen. But Springheel has his point of having them in an extra folder.

 

I'm a bit torn. (Maybe greebo can just add a toggle to DR "Hide LOD models"

 

Anyway, my priority now is creating a few decal LOD stages as I desp. need them

[Springheel]

@Badcogg: All your other points are somewhat moot, because the action of adding a model is very very seldom done (we have added only a handfull models since v1.0), but the action of "selecting a model in DR" is very often done (as is selecting an entity).

 

I agree with this.

 

I have no particular preference atm, so your idea can be done from my POV. Although Badcogg has a point with the "exit the current deep hirarchy and go up a lot and then down a lot just to find the first LOD stage".

 

 

I don't have a strong preference either, as long as they're not mixed in with regular models. But I didn't think mappers would generally be needing to find lod models--they'd primarily be called by entities, right?