Jump to content


Photo

TDM Engine Wishlist

engine

  • Please log in to reply
29 replies to this topic

#1 RPGista

RPGista

    Advanced Member

  • Member
  • PipPipPip
  • 1590 posts

Posted 22 August 2018 - 09:41 PM

Ok so we are having a lot of talk surrounding TDM's engine, and its amazing that we are having so much done in the way of improvements to it, lately. 

 

I thought it could be fun to have a (yet another) place where we as fans could post things we would like to see added to the game,  more specifically to its mechanics. Just suggestions for things that we feel would reinforce the game's strengths and design choices. Obviously, its up to the team and coders to know where to best place the resources and what needs to be prioritised. But a man can dream...

 

Having said that, lets start with something simple and easy...

 

 

Physics overhaul

 

 

Ok, maybe not an overhaul, but if theres one thing I always wanted to see improved in TDM (idtech4), before any graphical enhancement, was actually the physics. Not just the sort of fixing that has been taking place on specific points (like being able to KO guards with thrown or fallen objects, which was implemented back in the day). In my dream scenario, TDM would be going closer to a true "physics simulation" type of plataform (keeping gameplay needs at the front, of course). Why do I say that? The potential for a slow, deliberate stealth game in a trully responsive world are immense. Lets remember some of the possibilities to gameplay coming from great physics, as seen in Half Life 2 (I use it because its from the same generation as Doom3):

 

 

 

(In the second video) The moment where the player accidentally pushes the pc monitor and it falls down (at 2:02) making a mess, or the whole section from 2:50 to around 4:50 are great examples of things that would be so fun to have in TDM with a more advanced physics system:

 

- responsive chain reactions between objects, rewarding the use of moveables in maps because they can play a crucial role (you dont wanna accidentaly drop a bunch of bottles by bumping clumsily into the table holding them); 

 

- pushing objects into place to create obstacles and have their combined weight slow down enemies;

 

- destructable geometry, based on their material (this is so amazing, as you can see in the first video with the wood structures - they seem to be true breakable shapes, organically reacting to impacts or pressure). 

 

- being able to throw or cause the movement of heavy bodies that take out enemies and damage other bodies upon impact, even objects with so much mass that they can crush anything on their path; 

 

HL2 is a prime example because they used physics masterfully to create all sorts of interesting interactions between the player and the storyline, like puzzles you had to solve using objects and force, or using physics to set traps and take people out, etc.

 

Unfortunately, our engine, although I think many of the principles are there, seems a lot more limited overall. So I wonder how realistic it is for TDM to eventually improve its physics mechanics, either by enhancing doom3's own physics, or by implementing parts of external engines like Bullet or something. Im no expert and have absolutely no idea how hard any of this would be. But I do feel this could be something really interesting for future gameplay.


Edited by RPGista, 22 August 2018 - 09:47 PM.

  • Shadowthief likes this

#2 jaxa

jaxa

    Advanced Member

  • Member
  • PipPipPip
  • 1325 posts

Posted 22 August 2018 - 09:47 PM

Switch to Vulkan, use raytracing.



#3 HMart

HMart

    Advanced Member

  • Member
  • PipPipPip
  • 708 posts

Posted 23 August 2018 - 08:00 AM

The only way TDM engine will have improved physics, is if someone totally reworks the included physics engine, i assume not a easy task, or implements a totally new physics engine, personally i would recommend Newton Game Dynamics the same physics engine used in all Frictional Games, games, from what i heard is more accurate and stable than Havok, PhysX or Bullet the later ones are made for quantity not accuracy.

 

Vulkan is now a real possibility, because there's already a oficial vulkan renderer for idtech 4 and is also, one of the greatest possibilities to improve mission complexity, because of the massive savings on draw calls but for that to happen vulkan needs to be the only API available for TDM, if OpenGL is still a option, missions will need to be made thinking on it and so limited in scope has always. Vulkan does need modern dx12 GPUs so before making so radical change, a survey of the community would need to be done to see how many are still on dx11 and dx9 GPUs. 

 

Imo forget raytracing, unless you are using hybrid raytracing like D3D12 raytracing or RTX, normal raytracing is impractical for real time games, and from the look of things performance of RTX is not that impressive taking into account the price and power of the GPUs presented.

 

In the near future i would just implement stuff that is already included in some fork's of idtech 4.

 

True 64 bit HDR lighting with adaptive filmic tone mapping and gamma-correct rendering in linear RGB space. (is not the same has bloom).

Enhanced Subpixel Morphological Antialiasing

Half-Lambert lighting and rim lighting (used for characters in HL2)

Shadow maps only or Shadow maps along side the stencil shadows, i mean the ability to use both at the same time, so you can have alpha textures cast shadows even tho you still want to use exclusively soft stencil shadows for the rest.

Parallax occlusion mapping (is this not available already in TDM?)

SSAO and SSIL   - ssao (screen space ambient occlusion) and ssil (screen space indirect lighting)

 

In the future if possible:

 

Light Rays/Light Shafts

Dynamic liquid surfaces (remember FEAR water?) 

Volumetric particles, ability for shadows to cast unto particles in the air.

Terrain system

Vegetation system

PBR - Physical Based Rendering 


Edited by HMart, 23 August 2018 - 08:02 AM.

  • Shadowthief likes this

#4 cabalistic

cabalistic

    Member

  • Development Role
  • PipPip
  • 379 posts

Posted 23 August 2018 - 09:00 AM

You don't need Vulkan to reduce the draw calls; much of that is possible with modern OpenGL, as well. I've already done some experiments on it.

However, the major blocker right now is that TDM is still stuck on a very old GL version using ARB shaders, and these are not compatible at all with those modern techniques. So first we need to migrate *all* shaders to a newer GLSL version, and *then* we can potentially optimise some aspects with GL4+ techniques. That is a far goal, but nothing that's going to happen any time soon :)


  • Anderson likes this

#5 jaxa

jaxa

    Advanced Member

  • Member
  • PipPipPip
  • 1325 posts

Posted 23 August 2018 - 12:03 PM

Imo forget raytracing, unless you are using hybrid raytracing like D3D12 raytracing or RTX, normal raytracing is impractical for real time games, and from the look of things performance of RTX is not that impressive taking into account the price and power of the GPUs presented.

 

Considering I made the RTX thread, yeah, I'm talking about hybrid raytracing.

 

Also, I wouldn't expect TDM to implement raytracing within the next 3-5 years even though I suggested it. Meaning that by the time you would see it implemented, there would be at least another 2-3 generations of GPUs from the usual suspects, as well as GPUs from Intel.

 

HDR is a good one, definitely made for this game.

 

"Vegetation system". Can't that be accomplished with SEED? However, it would be neat to see something like this in DarkRadiant or a 3D modeling program.



#6 HMart

HMart

    Advanced Member

  • Member
  • PipPipPip
  • 708 posts

Posted 23 August 2018 - 02:32 PM

You don't need Vulkan to reduce the draw calls; much of that is possible with modern OpenGL, as well. I've already done some experiments on it.

However, the major blocker right now is that TDM is still stuck on a very old GL version using ARB shaders, and these are not compatible at all with those modern techniques. So first we need to migrate *all* shaders to a newer GLSL version, and *then* we can potentially optimise some aspects with GL4+ techniques. That is a far goal, but nothing that's going to happen any time soon :)

 

Vulkan does reduce the draw calls more than modern OpenGL ever could, but you are right modern OpenGL could be a viable option, this does mean that users will need at least DX11 GPUs, dx9 users will need to update.

If TDM engine is still using old ARB shaders then imo the priority should be to change those with GLSL versions indeed, i was under the impression that was already done.

 

 

Considering I made the RTX thread, yeah, I'm talking about hybrid raytracing.

 

Also, I wouldn't expect TDM to implement raytracing within the next 3-5 years even though I suggested it. Meaning that by the time you would see it implemented, there would be at least another 2-3 generations of GPUs from the usual suspects, as well as GPUs from Intel.

 

HDR is a good one, definitely made for this game.

 

"Vegetation system". Can't that be accomplished with SEED? However, it would be neat to see something like this in DarkRadiant or a 3D modeling program.

 

 I never used SEED so i can't really comment on it, but what i'm talking about is something like this, http://blog.wolfire....10/10/Imposters. This would be a system that could come after the terrain system for example. 


Edited by HMart, 23 August 2018 - 02:33 PM.


#7 Abusimplea

Abusimplea

    Member

  • Member
  • PipPip
  • 471 posts

Posted 23 August 2018 - 07:22 PM

My wishlist:

 

- Less soft bugs leading to randomly dropped tris (happens on dmap), odd behaviour of geometry when placed "too far" from the map origin (precision problem)

 

- A general solution to z-fights (this is the number one bug i encounter in betatesting) - maybe displacing geometry with z-fighting faces a tiny bit in dmap.

 

- The mentioned physics overhaul (hoping that it will lead to movers beeing able to move things that move things that move things...)

 

- A general solution for interior leaks leading to guards and their actions sounding like they come from the same room ewhile they are not even at the same level (don't know if possible, but this bug is very common in current TDM missions)

 

- Make everything have fresnel -> physics-based rendering (will probably not happen, but it's a wishlist)

 

Most of this are mapper QoL improvements because mappers will make more maps if making them involves less hassle.



#8 stumpy

stumpy

    Advanced Member

  • Member
  • PipPipPip
  • 1846 posts

Posted 23 August 2018 - 09:20 PM

movers do move things if you bind what you want moving to them.



#9 NeonsStyle

NeonsStyle

    Advanced Member

  • Member
  • PipPipPip
  • 1416 posts

Posted 23 August 2018 - 09:50 PM

1. For me. In physics, I'd like to be able to have a hanging ragdoll which didn't glitch out, and behaved correctly.

 

2. Being able to layer textures: ie Lay down one texture, then the following one, being able to adjust the transparency in editor

of following ones. This would let us have a far wider variety of effects on textures than just our limited set of decals. ie Stone wall;

layered with moss, vines. Currently, we'd either need that texture, or decals for both the final layers. Being able to layer in a more

broad way would be amazing. 

 

3. Being able to add cut scenes to an in game level, ie after the level has started. That'd be wicked. 


I have an eclectic YouTube channel making videos on a variety of games. Come and have look here:
 
https://www.youtube.com/c/NeonsStyleHD

 

Dark Mod Missions: Briarwood Manor - available here or in game

http://forums.thedar...2017-update-16/

 

 

 


#10 Obsttorte

Obsttorte

    Scripting guru, Mapper

  • Active Developer
  • PipPipPipPipPip
  • 5682 posts

Posted 24 August 2018 - 01:04 AM

My wishlist:
 
- A general solution to z-fights (this is the number one bug i encounter in betatesting) - maybe displacing geometry with z-fighting faces a tiny bit in dmap.

- A general solution for interior leaks leading to guards and their actions sounding like they come from the same room ewhile they are not even at the same level (don't know if possible, but this bug is very common in current TDM missions)

These are not bugs but the result of mappers mistakes. You can't blame the tool for not using it properly.
 
 
 

2. Being able to layer textures: ie Lay down one texture, then the following one, being able to adjust the transparency in editor
of following ones. This would let us have a far wider variety of effects on textures than just our limited set of decals. ie Stone wall;
layered with moss, vines. Currently, we'd either need that texture, or decals for both the final layers. Being able to layer in a more
broad way would be amazing. 
 
3. Being able to add cut scenes to an in game level, ie after the level has started. That'd be wicked.

2. That's already possible. You need custom material files for this, though. AFIAK there is no limit on how many layers you define there, although it might make sense to not overstress it. ;)

3. That's possible via a gui overlay. If I am not mistaken you can even pause the game during the cutscene is running.
  • Judith, Anderson and NeonsStyle like this
FM's: Builder Roads, Old Habits, Old Habits Rebuild
WIP's: Several. Although after playing Thief 4 I really wanna make a city mission.
Mapping and Scripting: Apples and Peaches
Sculptris Models and Tutorials: Obsttortes Models
My wiki articles: Obstipedia
Let's Map TDM YouTube playlist: ObstlerTube
Texture Blending in DR: DR ASE Blend Exporter

End of shameless self promotion.

#11 Judith

Judith

    Advanced Member

  • Member
  • PipPipPip
  • 1598 posts

Posted 24 August 2018 - 01:15 AM

Grayman did the whole wiki section on cameras, and they work, they just require rather complicated setup and scripting. It would be awesome to have a GUI editor for that, but it's not like it can't be done now.



#12 cabalistic

cabalistic

    Member

  • Development Role
  • PipPip
  • 379 posts

Posted 24 August 2018 - 02:27 AM

 

Vulkan does reduce the draw calls more than modern OpenGL ever could, but you are right modern OpenGL could be a viable option, this does mean that users will need at least DX11 GPUs, dx9 users will need to update.

If TDM engine is still using old ARB shaders then imo the priority should be to change those with GLSL versions indeed, i was under the impression that was already done.

 

The point is that a Vulkan renderer is just not very realistic right now. It would be a complete rewrite of the backend, and no, it doesn't help much that there is an implementation for Doom3 BFG. Keep in mind that TDM is based on the old Doom3 engine, not BFG. There are significant differences in the renderer so that a port is not nearly as straight-forward as you'd think. Besides, so far TDM has always tried to support as much older hardware as possible, so a Vulkan only renderer is not going to happen any time soon. Which would mean that it would need to keep maintained in addition to a GL renderer, which means additional maintenance overhead for an already fairly small team :)

 

What is much more realistic and something I'm personally pushing for is upgrading the current GL renderer to GL 3. This does mean getting rid of the remaining ARB shaders (which is tricky, because missions can in principle ship their own ARB shaders which are outside the direct control of the game engine) and removing the remaining fixed function stuff, which is still a lot of work. But once that is done, adding in optional GL4 optimizations based on availability is much simpler to do and maintain than a Vulkan renderer at the moment.


  • HMart and Anderson like this

#13 Judith

Judith

    Advanced Member

  • Member
  • PipPipPip
  • 1598 posts

Posted 24 August 2018 - 03:18 AM

Having a higher drawcall limit would be awesome, as much as better shadow performance, with both baked / static shadows, and dynamic shadows for selected entities (deferred and forward light rendering). It would be great if that was the priority, but I bet it's tons of difficult work.

 

As for shaders, I don't mind "classic" shaders instead of PBR, as you can still do a lot with them. PBR would require reworking all textures and materials, which might be a necessary decision to make at some point anyway. I'd argue that progress is more important than supporting missions from early days, but if that's the case, the last compatible version of "legacy TDM" should be available somewhere too.


Edited by Judith, 24 August 2018 - 03:23 AM.


#14 NeonsStyle

NeonsStyle

    Advanced Member

  • Member
  • PipPipPip
  • 1416 posts

Posted 24 August 2018 - 04:55 AM

The other thing I'd love to see in the game is water physics; so you can have water behave as expected, rather than just

water textures. You could make some really cool realistic waterfalls, beach scenes etc with that. Creating Surf albiet small

is almost impossible atm I think.


I have an eclectic YouTube channel making videos on a variety of games. Come and have look here:
 
https://www.youtube.com/c/NeonsStyleHD

 

Dark Mod Missions: Briarwood Manor - available here or in game

http://forums.thedar...2017-update-16/

 

 

 


#15 wesp5

wesp5

    Advanced Member

  • Member
  • PipPipPip
  • 638 posts

Posted 24 August 2018 - 05:50 AM

 

What is much more realistic and something I'm personally pushing for is upgrading the current GL renderer to GL 3. This does mean getting rid of the remaining ARB shaders (which is tricky, because missions can in principle ship their own ARB shaders which are outside the direct control of the game engine) and removing the remaining fixed function stuff, which is still a lot of work. But once that is done, adding in optional GL4 optimizations based on availability is much simpler to do and maintain than a Vulkan renderer at the moment.

This sounds like a good plan to me. I have the impression that a lot is going on regarding the TDM engine, so why not go this route and maybe concentrate on actually doing one new TDM version. It seems there are a BFG edition and a VR edition in the works as well...


Edited by wesp5, 24 August 2018 - 05:54 AM.

  • joma and Anderson like this

#16 Abusimplea

Abusimplea

    Member

  • Member
  • PipPip
  • 471 posts

Posted 24 August 2018 - 12:25 PM

movers do move things if you bind what you want moving to them.

My wishlist item was more about "non-preplanned" behaviour like when a crate gets moved away by a guard and reaches another crate wich then gets moved away by the first crate that is moved by the guard - if the guard has enough strength to move em both.
You certainly know about the double movable door block: Just queue up two movables behind a door and that door cant be opened by any guard because impulse is not transferred between the two moveables if only one of them is pushed by the door.
You can't easily build reliable traps based on moving walls or ceilings without banning movables in the trap area because of this.
 

These are not bugs but the result of mappers mistakes. You can't blame the tool for not using it properly.

I know, that this two are not engine bugs. A general fix for these would still make (less experienced) mappers lifes easier.


  • Anderson likes this

#17 stumpy

stumpy

    Advanced Member

  • Member
  • PipPipPip
  • 1846 posts

Posted 24 August 2018 - 02:49 PM

half-life 2 used a heavily modified havok physics engine.



#18 Obsttorte

Obsttorte

    Scripting guru, Mapper

  • Active Developer
  • PipPipPipPipPip
  • 5682 posts

Posted 24 August 2018 - 04:06 PM

I know, that this two are not engine bugs. A general fix for these would still make (less experienced) mappers lifes easier.

You cannot fix what isn't broken!
  • Anderson likes this
FM's: Builder Roads, Old Habits, Old Habits Rebuild
WIP's: Several. Although after playing Thief 4 I really wanna make a city mission.
Mapping and Scripting: Apples and Peaches
Sculptris Models and Tutorials: Obsttortes Models
My wiki articles: Obstipedia
Let's Map TDM YouTube playlist: ObstlerTube
Texture Blending in DR: DR ASE Blend Exporter

End of shameless self promotion.

#19 Boiler's_hiss

Boiler's_hiss

    Member

  • Member
  • PipPip
  • 27 posts

Posted 25 August 2018 - 03:05 AM

Facial animations generation by given text vowels/syllables/expression arguments.
Something with final result similar to this.

Spoiler

 

Door opening with knob turning animation generation.


Edited by Boiler's_hiss, 25 August 2018 - 03:11 AM.


#20 Anderson

Anderson

    Advanced Member

  • Member
  • PipPipPip
  • 1082 posts

Posted 25 August 2018 - 04:56 AM

Facial animations generation by given text vowels/syllables/expression arguments.
Something with final result similar to this.

Spoiler

 

Door opening with knob turning animation generation.

 

http://bugs.thedarkm...iew.php?id=2247

 

http://bugs.thedarkm...iew.php?id=3605

 

I saw how facial animations on Unreal 3 engine in Red Orchestra 2 were WIP, but they decided to cut it because it's too much effort for too little gameplay change.

About the same parameters of improving the game as the cape physics from Batman Arkham Asylum/Arkham City. They wanted to have those too in Red Orchestra through the Soviet raincoats and the German smaller capes for snipers. But it turned out that this too requires lots of resources. Basically in Batman developers made about 1000 animations to have the cape flow through different movements of the PC flawlessly without glitching into the environment and such. So, that also didn't make it into the game.

 

Basically none of this happens without a dedicated animator/coder.


Edited by Anderson, 25 August 2018 - 04:56 AM.

 "I really perceive that vanity about which most men merely prate — the vanity of the human or temporal life. I live continually in a reverie of the future. I have no faith in human perfectibility. I think that human exertion will have no appreciable effect upon humanity. Man is now only more active — not more happy — nor more wise, than he was 6000 years ago. The result will never vary — and to suppose that it will, is to suppose that the foregone man has lived in vain — that the foregone time is but the rudiment of the future — that the myriads who have perished have not been upon equal footing with ourselves — nor are we with our posterity. I cannot agree to lose sight of man the individual, in man the mass."...

 

 

- 2 July 1844 letter to James Russell Lowell from Edgar Allan Poe.

 


#21 demagogue

demagogue

    Mod hero

  • Active Developer
  • PipPipPipPipPip
  • 5402 posts

Posted 25 August 2018 - 09:43 AM

My dream improvement is multiplayer.

 

I would really, really love to play our maps coop style with some other players, and then we could have a few thieves vs. guards levels while we're at it.

 

The little I looked into it, I could already see how difficult it'd be, considering how many different systems it touches, and at the deepest levels of organization. But if it's at all possible, IMO it'd be worth the work.


Posted Image

#22 lowenz

lowenz

    Advanced Member

  • Member
  • PipPipPip
  • 1884 posts

Posted 25 August 2018 - 11:06 AM

Achievable goal: Temporally Stable SSAO (maybe SSAO)

 

https://www.slidesha...e/ssdo-52585061


Edited by lowenz, 25 August 2018 - 11:07 AM.

  • Judith and HMart like this

Task is not so much to see what no one has yet seen but to think what nobody has yet thought about that which everybody see. - E.S.


#23 chakkman

chakkman

    Advanced Member

  • Member
  • PipPipPip
  • 1018 posts

Posted 25 August 2018 - 12:45 PM

The only wish i have is that the performance will be improved and CPU consumption will be reduced at some point. Everything else is really secondary for me. It's the major issue i see with the TDM engine.



#24 rich_is_bored

rich_is_bored

    Advanced Member

  • Member
  • PipPipPip
  • 883 posts

Posted 25 August 2018 - 03:17 PM

When I wrote my color grading shader the reference material I was using called for using 3D textures. I found code in TDM's source to support loading such images but no corresponding material keyword to invoke it. Having that keyword would have saved me a ton of time.


  • Bikerdude, HMart, RPGista and 1 other like this

#25 duzenko

duzenko

    Advanced Member

  • Active Developer
  • PipPipPip
  • 1632 posts

Posted 26 August 2018 - 12:35 PM

The only way TDM engine will have improved physics, is if someone totally reworks the included physics engine, i assume not a easy task, or implements a totally new physics engine, personally i would recommend Newton Game Dynamics the same physics engine used in all Frictional Games, games, from what i heard is more accurate and stable than Havok, PhysX or Bullet the later ones are made for quantity not accuracy.

 

Vulkan is now a real possibility, because there's already a oficial vulkan renderer for idtech 4 and is also, one of the greatest possibilities to improve mission complexity, because of the massive savings on draw calls but for that to happen vulkan needs to be the only API available for TDM, if OpenGL is still a option, missions will need to be made thinking on it and so limited in scope has always. Vulkan does need modern dx12 GPUs so before making so radical change, a survey of the community would need to be done to see how many are still on dx11 and dx9 GPUs. 

 

Imo forget raytracing, unless you are using hybrid raytracing like D3D12 raytracing or RTX, normal raytracing is impractical for real time games, and from the look of things performance of RTX is not that impressive taking into account the price and power of the GPUs presented.

 

In the near future i would just implement stuff that is already included in some fork's of idtech 4.

 

True 64 bit HDR lighting with adaptive filmic tone mapping and gamma-correct rendering in linear RGB space. (is not the same has bloom).

Enhanced Subpixel Morphological Antialiasing

Half-Lambert lighting and rim lighting (used for characters in HL2)

Shadow maps only or Shadow maps along side the stencil shadows, i mean the ability to use both at the same time, so you can have alpha textures cast shadows even tho you still want to use exclusively soft stencil shadows for the rest.

Parallax occlusion mapping (is this not available already in TDM?)

SSAO and SSIL   - ssao (screen space ambient occlusion) and ssil (screen space indirect lighting)

 

In the future if possible:

 

Light Rays/Light Shafts

Dynamic liquid surfaces (remember FEAR water?) 

Volumetric particles, ability for shadows to cast unto particles in the air.

Terrain system

Vegetation system

PBR - Physical Based Rendering 

2.07 has shadow maps but do we have a single transparent shadow casting entity to make use of it?







Also tagged with one or more of these keywords: engine

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users