Jump to content


Photo

fhDOOM engine improvements

engine feature graphics improvement opengl glsl lighting shadows parallax shadowmapping

  • Please log in to reply
72 replies to this topic

#26 Obsttorte

Obsttorte

    Scripting guru, Mapper

  • Active Developer
  • PipPipPipPipPip
  • 5026 posts

Posted 11 May 2017 - 12:37 PM

Thanks HMart, but I know this one. It uses a heightmap.

 

I was more asking for the algorithm that uses the normalmap only to create this effect without any big performance impact as described by Mircae


Also it's fair to mention that some normal map textures in Xon do contain a grayscale bump map on the alpha channel too.

That is probably the heightmap. It is skalar and does only require one color channel. It is a typical approach to put the heightmap in the alpha channel of the normal map, as this channel is not used. Our normalmap images don't even contain one.


FM's: Builder Roads, Old Habits, Old Habits Rebuild
WIP's: Several. Although after playing Thief 4 I really wanna make a city mission.
Mapping and Scripting: Apples and Peaches
Sculptris Models and Tutorials: Obsttortes Models
My wiki articles: Obstipedia
Let's Map TDM YouTube playlist: ObstlerTube
Texture Blending in DR: DR ASE Blend Exporter

End of shameless self promotion.

#27 nbohr1more

nbohr1more

    Darkmod PR, Wordsmith

  • Development Role
  • PipPipPipPipPip
  • 7572 posts

Posted 11 May 2017 - 12:53 PM

LOL.

 

I guess POM is now obsolete. The new hotness is Quad-tree Displacement Mapping:

 

http://docplayer.net...t-blending.html


Please visit TDM's IndieDB site and help promote the mod:

http://www.indiedb.c...ds/the-dark-mod

(Yeah, shameless promotion... but traffic is traffic folks...)

#28 HMart

HMart

    Advanced Member

  • Member
  • PipPipPip
  • 521 posts

Posted 11 May 2017 - 01:26 PM

Thanks HMart, but I know this one. It uses a heightmap.
 
I was more asking for the algorithm that uses the normalmap only to create this effect without any big performance impact as described by Mircae
 
That is probably the heightmap. It is skalar and does only require one color channel. It is a typical approach to put the heightmap in the alpha channel of the normal map, as this channel is not used. Our normalmap images don't even contain one.


Ha ok my bad, about using only the normal map for POM (or even PM) IMO the only way i see, is putting the bump map on the alpha channel or somehow improve the info on the blue channel, if someone is claiming they can make good parallax info from a "normal" normal map, and only use the green and red channel like most normal maps are, i also want to know how they do it. 
 

LOL.
 
I guess POM is now obsolete. The new hotness is Quad-tree Displacement Mapping:
 
http://docplayer.net...t-blending.html


lol yes there's plenty of (virtual displacement mapping techniques), btw the "Two Worlds 2" guys, claim its faster than POM but i don't know of any other game where that technique is implemented, and i never played that game to know if is true, perhaps some patent problems are preventing it from spreading? 


Edited by HMart, 11 May 2017 - 01:32 PM.


#29 MirceaKitsune

MirceaKitsune

    Advanced Member

  • Member
  • PipPipPip
  • 590 posts

Posted 11 May 2017 - 02:15 PM

Sorry for the confusion about "which type of parallax mapping I mean": To be fair, I'm not sure myself which is which: I know that parallax is the effect that, alongside bump mapping which creates directional shadows on the details of a texture, stretches and shrinks parts of a texture based on camera position / angle in order to create the illusion of depth. I imagine there are multiple ways of achieving that, and personally I'm not picky about which is used... the most realistic is obviously the best, but I assume performance also plays an important role.



#30 Obsttorte

Obsttorte

    Scripting guru, Mapper

  • Active Developer
  • PipPipPipPipPip
  • 5026 posts

Posted 11 May 2017 - 02:48 PM


somehow improve the info on the blue channel

Isn't the blue channel only storing the z-component of the normal?


FM's: Builder Roads, Old Habits, Old Habits Rebuild
WIP's: Several. Although after playing Thief 4 I really wanna make a city mission.
Mapping and Scripting: Apples and Peaches
Sculptris Models and Tutorials: Obsttortes Models
My wiki articles: Obstipedia
Let's Map TDM YouTube playlist: ObstlerTube
Texture Blending in DR: DR ASE Blend Exporter

End of shameless self promotion.

#31 HMart

HMart

    Advanced Member

  • Member
  • PipPipPip
  • 521 posts

Posted 11 May 2017 - 10:27 PM

Sorry for the confusion about "which type of parallax mapping I mean": To be fair, I'm not sure myself which is which: I know that parallax is the effect that, alongside bump mapping which creates directional shadows on the details of a texture, stretches and shrinks parts of a texture based on camera position / angle in order to create the illusion of depth. I imagine there are multiple ways of achieving that, and personally I'm not picky about which is used... the most realistic is obviously the best, but I assume performance also plays an important role.

 

No problem and sorry if i came across has a douch, btw normal mapping can also technically be considered bump mapping, both just simulate "bumps" on a 2D surface, bump mapping using a B&W texture was the first tech invented to simulate "bumps" (another very old tech that i don't seam to remember the name now, also made that but made everything look like shiny glass from any angle), then normal mapping was invented, visually a bump map and a normal map do the exact same thing, the diference, is that the B&W bump map holds extra info that was not really needed to simulate simple bumps, it has the ability to hold complex height info about the surface in all three components, x y z (that is why it is still used today for Tessellation for example), making the math more heavy then it needed to be, a normal map works just by using the normal's of the surface, simplifying the math (there the term normal mapping) and only on the x y components and making z (height) always be 1, visually this made no diference but changed everything on the performance front, so yes, performance does play a major role in the choosing of what technique to implement, specially those that make heavy use of ray tracing, like Self Shadowing Parallax Occlusion Mapping, Relief Mapping, Steep Parallax Mapping, etc, etc, so imo referring to the techniques by their names prevent confusion and headaches in the long run.  
 

Also IMO both Parallax Mapping and Parallax Occlusion Mapping could be implemented, used perhaps has different LOD stages or different game quality stages, they can be coded to work with the same bump map, but one works on older PC's and is faster, but looks worse (at grazing angles), the other looks better but is slower and only run on "modern" shader model 3.0 GPU's and beyond.   

 

Isn't the blue channel only storing the z-component of the normal?

 
Yes but afaik is just storing a flat height of 1 and most of the time the z-component is not even needed to do correct normal mapping, there the blue channel not being even used in some games, so, from what i know, a simple normal map, do not stores enough info to make a good B&W bump map (also called height map), but some do try to do it with varied success, i just don't know exactly how they do it, would love to know tho. 


Edited by HMart, 11 May 2017 - 10:31 PM.


#32 Obsttorte

Obsttorte

    Scripting guru, Mapper

  • Active Developer
  • PipPipPipPipPip
  • 5026 posts

Posted 12 May 2017 - 05:14 AM

For Reference:

Iterative Parallex Mapping (with slope information)

 

Similar cheap like Parallex mapping, but better in regards to visuals. Fixed amount of iterations, so no branching needed (-> can be implemented in arb shader).

 

Cone Step Mapping

 

The results look good, but it requires way more precomputed data than Parallax methods

 

Steep Parallax Mapping

 

The first one seems the most feasable in regards to implementation.


  • HMart and Epifire like this
FM's: Builder Roads, Old Habits, Old Habits Rebuild
WIP's: Several. Although after playing Thief 4 I really wanna make a city mission.
Mapping and Scripting: Apples and Peaches
Sculptris Models and Tutorials: Obsttortes Models
My wiki articles: Obstipedia
Let's Map TDM YouTube playlist: ObstlerTube
Texture Blending in DR: DR ASE Blend Exporter

End of shameless self promotion.

#33 HMart

HMart

    Advanced Member

  • Member
  • PipPipPip
  • 521 posts

Posted 12 May 2017 - 07:06 AM

That parallax mapping technique on the first link with offset limiting seams to solve the grazing angle problem pretty nicely!  IMO one good bullet point to chose this PM tech over others, performance is all that matters if the visuals are good enough.



#34 MirceaKitsune

MirceaKitsune

    Advanced Member

  • Member
  • PipPipPip
  • 590 posts

Posted 12 May 2017 - 08:14 AM

Steep mapping looks like what Xonotic does, since it definitely supports that full depth effect. It would certainly be nice if we could see it for TDM as well... imagine that depth effect showing in between bricks on walls :D



#35 HMart

HMart

    Advanced Member

  • Member
  • PipPipPip
  • 521 posts

Posted 12 May 2017 - 05:12 PM

Steep mapping looks like what Xonotic does, since it definitely supports that full depth effect. It would certainly be nice if we could see it for TDM as well... imagine that depth effect showing in between bricks on walls :D

 

Yes steep parallax mapping is a very nice technique and one of the most realistic around specially because it can do self shadowing on the bumps, the problem, is a very performance hungry technique, IMO POM (without self shadowing) looks awesome and is faster than steep Parallax Mapping. In the end IMO for a game like TDM where performance is critical, that PM with offset limiting seams the best option, today PM is considered to be almost as fast as normal mapping and it fakes geom better than it (when it works fine).

 

Btw :o

 

 

 

I'm sure this kills performance more than x64 Tesselation but is impressive how much you can fake today with just texture tricks and cleaver math. 



#36 Obsttorte

Obsttorte

    Scripting guru, Mapper

  • Active Developer
  • PipPipPipPipPip
  • 5026 posts

Posted 13 May 2017 - 12:52 AM

The more advanced variants aren't really an option for us anyway as long as we don't have GLSL support. With arb shaders we are a bit limited about which shaders we can implement.


  • Epifire likes this
FM's: Builder Roads, Old Habits, Old Habits Rebuild
WIP's: Several. Although after playing Thief 4 I really wanna make a city mission.
Mapping and Scripting: Apples and Peaches
Sculptris Models and Tutorials: Obsttortes Models
My wiki articles: Obstipedia
Let's Map TDM YouTube playlist: ObstlerTube
Texture Blending in DR: DR ASE Blend Exporter

End of shameless self promotion.

#37 Epifire

Epifire

    Member

  • Active Developer
  • PipPip
  • 354 posts

Posted 13 May 2017 - 02:52 AM

The more advanced variants aren't really an option for us anyway as long as we don't have GLSL support. With arb shaders we are a bit limited about which shaders we can implement.

 

Even if it was a simple parallax shader I would be very intent on using that myself. If you fine fellows ever find the itch to take a shot at that, I'd be down for making some new materials to test it with.  :D


Needing dem environment models?

 

Speak to the EPI!

 

Or rummage my online mesh repository instead...  https://www.mediafire.com/folder/7q1r8him1y63d/TDM%20Repository


#38 HMart

HMart

    Advanced Member

  • Member
  • PipPipPip
  • 521 posts

Posted 13 May 2017 - 08:01 AM

The more advanced variants aren't really an option for us anyway as long as we don't have GLSL support. With arb shaders we are a bit limited about which shaders we can implement.

 

I think some of you guys underestimate the engine TDM is using, a Steep Parallax Mapping mod was made for original Doom 3 and that game uses ARB and OpenGL 1.5, Sikkmod also implemented on original Doom 3 plenty of advanced shaders, including, Bloom, SSAO, HDAO and others, Parallax Occlusion Mapping, HDR lighting, etc, etc, guys the engine is better then you think, sometimes we just need to give it a small push. :)    

 

 

Even if it was a simple parallax shader I would be very intent on using that myself. If you fine fellows ever find the itch to take a shot at that, I'd be down for making some new materials to test it with.  :D

 

Perhaps i'm recalling wrong but i seam to remember someone on this forum showing Parallax Occlusion Mapping working on TDM, just don't remember who or what thread. 

 

I never tried and i don't bet it will work but you could try to substitute the TDM interaction shader by one of the Doom 3 Sikkmod interactions shaders and see what it does, there is at lest one with POM, if it works you can test your bump maps with it, until someone implements POM natively into TDM. 



#39 Springheel

Springheel

    Creative Director (retired)

  • Admin
  • 36232 posts

Posted 13 May 2017 - 08:08 AM

Sikkmod also implemented on original Doom 3 plenty of advanced shaders, including, Bloom, SSAO, HDAO and others, Parallax Occlusion Mapping, HDR lighting, etc, etc, guys the engine is better then you think, sometimes we just need to give it a small push.

 

 

We have played with the Sikkmod changes before, but the performance impact is huge.



#40 MirceaKitsune

MirceaKitsune

    Advanced Member

  • Member
  • PipPipPip
  • 590 posts

Posted 13 May 2017 - 08:12 AM

I don't underestimate the idTech4 engine, although I'm seeing a lot of people who are. For an engine that was developed 10 years ago, the features and flexibility and ingenuity it offers are remarkable! The community made some amazing engines based on itTech 1 to 3... imagine what we could do with a heavily modernized version of 4, we'd be competing with Unreal and Unity! Sadly the engine does have a lot of aged components, as it uses the cutting edge technology that was available back in 2005 (ish), which is the reason why improvements like this are important.



#41 HMart

HMart

    Advanced Member

  • Member
  • PipPipPip
  • 521 posts

Posted 13 May 2017 - 08:58 AM

 

We have played with the Sikkmod changes before, but the performance impact is huge.

 

Yes some of the effects in Sikkmod are very demanding but imo has more to do with the effects them self's or how they were written then the inability of the engine to render them at good performance, i'm sure you know ARB performance is dependent on the GPU more then anything. 

 

I don't underestimate the idTech4 engine, although I'm seeing a lot of people who are. For an engine that was developed 10 years ago, the features and flexibility and ingenuity it offers are remarkable! The community made some amazing engines based on itTech 1 to 3... imagine what we could do with a heavily modernized version of 4, we'd be competing with Unreal and Unity! Sadly the engine does have a lot of aged components, as it uses the cutting edge technology that was available back in 2005 (ish), which is the reason why improvements like this are important.

 

Vanilla idtech 4 is indeed old by today standards and relies a bit to much on the CPU side of things and is single-threaded, where modern engines are more GPU oriented and multi-threaded, it was just made at the time where CPU's (and single core ones) where more important than GPU's, apart from this obviously important limitations imo is a very advanced engine.

 

Also you have a "heavilly modernized" version of idtech 4 on the BFG version, but i would be impressed if the TDM team would risk such a jump of engines. 



#42 Obsttorte

Obsttorte

    Scripting guru, Mapper

  • Active Developer
  • PipPipPipPipPip
  • 5026 posts

Posted 13 May 2017 - 12:10 PM


i'm sure you know ARB performance is dependent on the GPU more then anything.

That is obvious. But using ARB instead of GLSL makes things even worse. Sikkmod had to hack a lot of stuff back in the day when I fiddled with merging it with tdm, as the source code wasn't released back than. I don't know whether it ever got updated, though.


Sikkmod also implemented on original Doom 3 plenty of advanced shaders, including, Bloom, SSAO, HDAO and others, Parallax Occlusion Mapping, HDR lighting, etc, etc

We already have Bloom, the effect of SSAO was neglectable but performance houngry. SSIL had no real effect either, as our textures don't use high specularity. I can't remember of POM but it was two or three years ago and HDR looked very artificial and overstylized (many members didn't like that effect).

 

I don't think anybody here underestimates the potential of idTech4. But people underestimate the amount of time and work it needs to implement such stuff. And that due to some limitations caused by the simple fact that the engine is old, even though the can be overcome, things don't get neccesarely better.


The more advanced variants aren't really an option for us anyway as long as we don't have GLSL support. With arb shaders we are a bit limited about which shaders we can implement.

We can of course implement advanced shaders with our current setup, but it won't be efficient. Doing this with algorithms that are already performance intensive like the more advanced versions of POM is simple not the best idea. Not to mention that coding in ARB is a pain in the arse for a script kiddie like me ;)


FM's: Builder Roads, Old Habits, Old Habits Rebuild
WIP's: Several. Although after playing Thief 4 I really wanna make a city mission.
Mapping and Scripting: Apples and Peaches
Sculptris Models and Tutorials: Obsttortes Models
My wiki articles: Obstipedia
Let's Map TDM YouTube playlist: ObstlerTube
Texture Blending in DR: DR ASE Blend Exporter

End of shameless self promotion.

#43 nbohr1more

nbohr1more

    Darkmod PR, Wordsmith

  • Development Role
  • PipPipPipPipPip
  • 7572 posts

Posted 13 May 2017 - 12:59 PM

We can create dedicated map types for POM (etc) so the performance can be improved. We can also add Pat Raynors GLSL backend if needed. Lots of options. The biggest fish to fry is GPU skinning. Everything is secondary to that.
Please visit TDM's IndieDB site and help promote the mod:

http://www.indiedb.c...ds/the-dark-mod

(Yeah, shameless promotion... but traffic is traffic folks...)

#44 Obsttorte

Obsttorte

    Scripting guru, Mapper

  • Active Developer
  • PipPipPipPipPip
  • 5026 posts

Posted 13 May 2017 - 01:14 PM

We=Grayman?! :)


FM's: Builder Roads, Old Habits, Old Habits Rebuild
WIP's: Several. Although after playing Thief 4 I really wanna make a city mission.
Mapping and Scripting: Apples and Peaches
Sculptris Models and Tutorials: Obsttortes Models
My wiki articles: Obstipedia
Let's Map TDM YouTube playlist: ObstlerTube
Texture Blending in DR: DR ASE Blend Exporter

End of shameless self promotion.

#45 MirceaKitsune

MirceaKitsune

    Advanced Member

  • Member
  • PipPipPip
  • 590 posts

Posted 13 May 2017 - 01:40 PM

Vanilla idtech 4 is indeed old by today standards and relies a bit to much on the CPU side of things and is single-threaded, where modern engines are more GPU oriented and multi-threaded, it was just made at the time where CPU's (and single core ones) where more important than GPU's, apart from this obviously important limitations imo is a very advanced engine.

 

Also you have a "heavilly modernized" version of idtech 4 on the BFG version, but i would be impressed if the TDM team would risk such a jump of engines. 

 

True there. Multiple CPU threads were experimental technology in the days of Doom 3, so it's easy to tell why multi-threading never made it. GLSL was also more limited I believe... I'm not sure if OpenGL 3.3 was even a standard at the time! These are all things which ideally, can be changed in the engine... but obviously I don't underestimate the amount of work that would take.

 

What I'm saying is that regardless of the very under-the-hood limitations, the infrastructure and layout of the engine is unusually modern to this day! That's referring to how flexibly entities can be defined and interconnected, how modular md5mesh / md5anim models are, the shader system which is capable of visual effects many modern engines still didn't add (like heat haze), the ingenious area portal system, and let's not forget that interactive touchscreens are a part of the engine (even if TDM sadly never made steampunk variants), also it comes with a fully functioning physics system plus ragdolls (which work just as well as Bullet / ODE).

 

There are modern FOSS engines which struggle to implement all of these things or better equivalents. Some open-source games that look really good only have basic physics (no objects pushing each other or rolling around), just the most standard shaders, every single entity and functionality is hardcoded and must be compiled by hand, geometry culling relies solely on surface hiding (no smart area portals), and interactive screens are a dream.


Edited by MirceaKitsune, 13 May 2017 - 01:41 PM.


#46 nbohr1more

nbohr1more

    Darkmod PR, Wordsmith

  • Development Role
  • PipPipPipPipPip
  • 7572 posts

Posted 13 May 2017 - 02:50 PM

We=Grayman?! :)


I know how to add map types to the renderer and I can copy the GLSL backend from the other project. I cannot write GLSL but SteveL can so if he comes back we have all the ingredients. I can't speak for Duzenko or Stgatilov but the both seem to have skills that exceed my own.
  • HMart likes this
Please visit TDM's IndieDB site and help promote the mod:

http://www.indiedb.c...ds/the-dark-mod

(Yeah, shameless promotion... but traffic is traffic folks...)

#47 duzenko

duzenko

    Member

  • Mission Beta Tester
  • PipPip
  • 375 posts

Posted 13 May 2017 - 03:21 PM

I would like to see the glsl support very much. Porting ARB to glsl can't be too hard.

Correct me if I'm wrong but the real problem is going to be replacing all legacy stuff like modelview and texture matrixes, texgens, global vs per vertex colors and normals, etc. I presume that GLSL backend includes all of that?



#48 nbohr1more

nbohr1more

    Darkmod PR, Wordsmith

  • Development Role
  • PipPipPipPipPip
  • 7572 posts

Posted 13 May 2017 - 05:08 PM

To my knowledge, Pat Raynor's work was still a hybrid of pre-2.0 stuff and GLSL kludged on the shading side (similar to how the ARB shaders are also kludged onto the fixed function pipeline). He created another project where BFG changes were incorporated which probably covers most of the modernizations you are referring to.

 

Edit:

 

Link to the new Pat Raynor stuff:

 

https://github.com/raynorpat/Doom3


Please visit TDM's IndieDB site and help promote the mod:

http://www.indiedb.c...ds/the-dark-mod

(Yeah, shameless promotion... but traffic is traffic folks...)

#49 duzenko

duzenko

    Member

  • Mission Beta Tester
  • PipPip
  • 375 posts

Posted 14 May 2017 - 01:50 AM

Curious: first thing I looked at the depth shader on things I did recently.

1) can't see code that would clip a mirror near plane.

2) can't see where the referenced clip() function is defined

3) how is #define implemented? AFAIK it's not standard GLSL

4) is there support for vegetation LOD alpha-test? Can't see it.

 

Git status is WIP but last change was year ago. How finished is it generally?



#50 duzenko

duzenko

    Member

  • Mission Beta Tester
  • PipPip
  • 375 posts

Posted 14 May 2017 - 04:43 PM

Heads up: I'm in the middle of adding GLSL support, depth stage as the first step.

It half works right now.


  • nbohr1more likes this





Also tagged with one or more of these keywords: engine, feature, graphics, improvement, opengl, glsl, lighting, shadows, parallax, shadowmapping

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users