Jump to content


Photo

TDM Engine Development Page

idtech4 development engine

  • Please log in to reply
856 replies to this topic

#26 Bikerdude

Bikerdude

    Mod hero

  • Member
  • PipPipPipPipPip
  • 19978 posts

Posted 15 June 2014 - 05:42 AM

So, Sikkpin has offered to help. Maybe we should prompt him for assistance sooner rather than later before he changes his mind? :)
Link for reference:http://forums.thedar...on-doom3-world/

How goes this nbohr..?

#27 nbohr1more

nbohr1more

    Darkmod PR, Wordsmith

  • Development Role
  • PipPipPipPipPip
  • 8872 posts

Posted 15 June 2014 - 05:47 AM

Compare the performance for overlapping lights. Depending on your hardware you may need to increase the light count to well over 50 lights to see the difference. Also, check the performance of those new graphic features you added in SVN.
Please visit TDM's IndieDB site and help promote the mod:

http://www.indiedb.c...ds/the-dark-mod

(Yeah, shameless promotion... but traffic is traffic folks...)

#28 Obsttorte

Obsttorte

    Scripting guru, Mapper

  • Active Developer
  • PipPipPipPipPip
  • 5609 posts

Posted 15 June 2014 - 07:01 AM

The changes does not seem to have an effect neither for sikkpins effects nor for high lightcounts. If it should really help with 50+ lights than this is fine, but pretty useless :(
FM's: Builder Roads, Old Habits, Old Habits Rebuild
WIP's: Several. Although after playing Thief 4 I really wanna make a city mission.
Mapping and Scripting: Apples and Peaches
Sculptris Models and Tutorials: Obsttortes Models
My wiki articles: Obstipedia
Let's Map TDM YouTube playlist: ObstlerTube
Texture Blending in DR: DR ASE Blend Exporter

End of shameless self promotion.

#29 New Horizon

New Horizon

    Mod hero

  • Active Developer
  • PipPipPipPipPip
  • 13815 posts

Posted 15 June 2014 - 07:17 AM

Looks like he updated his post with some fixes to get openmp partly working under windows. If I understand this correctly, openmp should allow certain functions to be shared between multiple cores on a CPU? If revelator has indeed gotten it working, that could potentially see a decent performance boost in TDM if it could offload to a second core.

The changes does not seem to have an effect neither for sikkpins effects nor for high lightcounts. If it should really help with 50+ lights than this is fine, but pretty useless :(


I suspect that the benefits would be somewhat minimal under TDM since we have a higher overhead than vanilla D3, but if he has gotten openmp working...then we may see some improvements yet.

#30 nbohr1more

nbohr1more

    Darkmod PR, Wordsmith

  • Development Role
  • PipPipPipPipPip
  • 8872 posts

Posted 15 June 2014 - 07:42 AM

Is this up on SVN? I'd like to check it out after work. The benefits may vary depending on your hardware and drivers etc. I suspect that it still relies on CPU skinning so you might not see much benefit if you aren't GPU limited.
Please visit TDM's IndieDB site and help promote the mod:

http://www.indiedb.c...ds/the-dark-mod

(Yeah, shameless promotion... but traffic is traffic folks...)

#31 New Horizon

New Horizon

    Mod hero

  • Active Developer
  • PipPipPipPipPip
  • 13815 posts

Posted 15 June 2014 - 07:47 AM

Is this up on SVN? I'd like to check it out after work. The benefits may vary depending on your hardware and drivers etc. I suspect that it still relies on CPU skinning so you might not see much benefit if you aren't GPU limited.


I don't believe it's on SVN yet, but I would also like to see how it all works. I'm just trying to get the build tools setup again here so I won't have time to merge it in until this evening. I'll be curious to see if openmp offers any help since I have a dual core system.

#32 nbohr1more

nbohr1more

    Darkmod PR, Wordsmith

  • Development Role
  • PipPipPipPipPip
  • 8872 posts

Posted 15 June 2014 - 09:50 AM

I doubt Relevator tried this without also having improved VBO support so his performance findings may be reliant
on having the new VBO code as well.

Mh's changes:

http://pastebin.com/rHrwP0nA

Relevator's codebase:

https://github.com/revelator/MHDoom
Please visit TDM's IndieDB site and help promote the mod:

http://www.indiedb.c...ds/the-dark-mod

(Yeah, shameless promotion... but traffic is traffic folks...)

#33 Obsttorte

Obsttorte

    Scripting guru, Mapper

  • Active Developer
  • PipPipPipPipPip
  • 5609 posts

Posted 15 June 2014 - 11:35 AM

Comitted. EDIT: The source.
FM's: Builder Roads, Old Habits, Old Habits Rebuild
WIP's: Several. Although after playing Thief 4 I really wanna make a city mission.
Mapping and Scripting: Apples and Peaches
Sculptris Models and Tutorials: Obsttortes Models
My wiki articles: Obstipedia
Let's Map TDM YouTube playlist: ObstlerTube
Texture Blending in DR: DR ASE Blend Exporter

End of shameless self promotion.

#34 New Horizon

New Horizon

    Mod hero

  • Active Developer
  • PipPipPipPipPip
  • 13815 posts

Posted 15 June 2014 - 06:33 PM

Does the commit include that last update from revelator? Compiled the source and tried it out, but sadly not seeing any perceivable performance gain. Maybe 1 fps, but that would be about it.

#35 demagogue

demagogue

    Mod hero

  • Active Developer
  • PipPipPipPipPip
  • 5382 posts

Posted 15 June 2014 - 09:11 PM

With your guy's experiences it's sounding like an optimization on paper that doesn't translate so much in practice, or maybe would for vanilla D3 but our greater overhead blunts it. But something significant may be buried somewhere inside of it yet.
Posted Image

#36 Obsttorte

Obsttorte

    Scripting guru, Mapper

  • Active Developer
  • PipPipPipPipPip
  • 5609 posts

Posted 16 June 2014 - 01:29 AM

Does the commit include that last update from revelator? Compiled the source and tried it out, but sadly not seeing any perceivable performance gain. Maybe 1 fps, but that would be about it.

Yes, but as said there is no noticeable performance gain.

I think it would be more benefitting to include openmp in the ai thinking process, as this would allow us to have more ai.
FM's: Builder Roads, Old Habits, Old Habits Rebuild
WIP's: Several. Although after playing Thief 4 I really wanna make a city mission.
Mapping and Scripting: Apples and Peaches
Sculptris Models and Tutorials: Obsttortes Models
My wiki articles: Obstipedia
Let's Map TDM YouTube playlist: ObstlerTube
Texture Blending in DR: DR ASE Blend Exporter

End of shameless self promotion.

#37 New Horizon

New Horizon

    Mod hero

  • Active Developer
  • PipPipPipPipPip
  • 13815 posts

Posted 16 June 2014 - 07:11 AM

I think it would be more benefitting to include openmp in the ai thinking process, as this would allow us to have more ai.


Could it be included in the lightgem too? Anything that could be split over multiple cores would likely be beneficial.

#38 Lux

Lux

    Advanced Member

  • Member
  • PipPipPip
  • 1575 posts

Posted 16 June 2014 - 07:26 AM

Could this also be used in some way to facilitate faster initial loading time?

#39 demagogue

demagogue

    Mod hero

  • Active Developer
  • PipPipPipPipPip
  • 5382 posts

Posted 16 June 2014 - 07:53 AM

It's a shadow rendering optimization, so I doubt that, but what do I know.
Posted Image

#40 nbohr1more

nbohr1more

    Darkmod PR, Wordsmith

  • Development Role
  • PipPipPipPipPip
  • 8872 posts

Posted 16 June 2014 - 08:05 AM

OpenMP can offer more than shadow optimization. My concern is that I think it's framework much like the Boost libraries (which also
have multi-core functions) which may make the codebase harder to read and maintain. The BFG codebase doesn't use a framework to grant multi-core
support and instead has explicit design features (job system) for it and therefore is much easier to maintain.

Neither option is really easy to implement but OpenMP is "easier" than the latter.
Please visit TDM's IndieDB site and help promote the mod:

http://www.indiedb.c...ds/the-dark-mod

(Yeah, shameless promotion... but traffic is traffic folks...)

#41 Obsttorte

Obsttorte

    Scripting guru, Mapper

  • Active Developer
  • PipPipPipPipPip
  • 5609 posts

Posted 16 June 2014 - 11:32 AM

Besides graphics the most performance costing aspect is the ai. If those guys are alerted, they are eating pretty much. If we would be able to multithread this aspect, thus meaning calling the think() methods of the relating classes in several threads, than this would allow for a higher ai count in FM's (depending on the number of cores, but even on my dual core cpu this may gives the ability of up to 50% more ai or so, roughly guessing).
  • Springheel and SteveL like this
FM's: Builder Roads, Old Habits, Old Habits Rebuild
WIP's: Several. Although after playing Thief 4 I really wanna make a city mission.
Mapping and Scripting: Apples and Peaches
Sculptris Models and Tutorials: Obsttortes Models
My wiki articles: Obstipedia
Let's Map TDM YouTube playlist: ObstlerTube
Texture Blending in DR: DR ASE Blend Exporter

End of shameless self promotion.

#42 motorsep

motorsep

    Advanced Member

  • Member
  • PipPipPip
  • 884 posts

Posted 16 June 2014 - 11:43 AM

Note that in BFG engine, entire game code is not threaded (physics, AI, etc.). It just runs on a separate thread from anything else, thus performance much better than idTech 4 game code.

RAGE on the other hand has AI and physics on separate threads (there maybe more or less separation going on, I am not 100% sure). It was hell of a work to thread it to my understanding, so I wouldn't count on having it done in the garage, especially using idTech 4.

#43 Obsttorte

Obsttorte

    Scripting guru, Mapper

  • Active Developer
  • PipPipPipPipPip
  • 5609 posts

Posted 16 June 2014 - 12:45 PM

I never said it is easy, just wanted to point out that it would be beneficial.
FM's: Builder Roads, Old Habits, Old Habits Rebuild
WIP's: Several. Although after playing Thief 4 I really wanna make a city mission.
Mapping and Scripting: Apples and Peaches
Sculptris Models and Tutorials: Obsttortes Models
My wiki articles: Obstipedia
Let's Map TDM YouTube playlist: ObstlerTube
Texture Blending in DR: DR ASE Blend Exporter

End of shameless self promotion.

#44 motorsep

motorsep

    Advanced Member

  • Member
  • PipPipPip
  • 884 posts

Posted 16 June 2014 - 01:43 PM

Looking at the rate of adoption of BFG engine, I'd say it's nearly impossible ;)

#45 Obsttorte

Obsttorte

    Scripting guru, Mapper

  • Active Developer
  • PipPipPipPipPip
  • 5609 posts

Posted 16 June 2014 - 01:45 PM

We'll see.


If we just had a couple more people skilled in programming which would be willing to help us.
FM's: Builder Roads, Old Habits, Old Habits Rebuild
WIP's: Several. Although after playing Thief 4 I really wanna make a city mission.
Mapping and Scripting: Apples and Peaches
Sculptris Models and Tutorials: Obsttortes Models
My wiki articles: Obstipedia
Let's Map TDM YouTube playlist: ObstlerTube
Texture Blending in DR: DR ASE Blend Exporter

End of shameless self promotion.

#46 nbohr1more

nbohr1more

    Darkmod PR, Wordsmith

  • Development Role
  • PipPipPipPipPip
  • 8872 posts

Posted 16 June 2014 - 04:57 PM

Looks like relevator is cracking away at the OpenMP fixes still.

http://forums.inside...&p=53063#p53063

Hopefully he can join the conversation soon... (problems with forum signup)
Please visit TDM's IndieDB site and help promote the mod:

http://www.indiedb.c...ds/the-dark-mod

(Yeah, shameless promotion... but traffic is traffic folks...)

#47 New Horizon

New Horizon

    Mod hero

  • Active Developer
  • PipPipPipPipPip
  • 13815 posts

Posted 16 June 2014 - 05:25 PM

Looks like relevator is cracking away at the OpenMP fixes still.

http://forums.inside...&p=53063#p53063

Hopefully he can join the conversation soon... (problems with forum signup)


Awesome stuff. I really hope he can crack this thing wide open so we can see some performance gains...although it seems like no matter what he tries, idtech4 is determined not to allow a relatively easy transition to better performance. I imagine if TDM were able to take advantage to both cores on my cpu that there would have to be a somewhat noticeable gain.

#48 nbohr1more

nbohr1more

    Darkmod PR, Wordsmith

  • Development Role
  • PipPipPipPipPip
  • 8872 posts

Posted 16 June 2014 - 09:35 PM

So, I wonder about the road not traveled.

We are so vested in the current monolithic TDM standalone build that any big "code merge" is met with some amount of apprehension.

The way I see it, the easiest way to determine the feasibility of such a thing would be to take v1.07 and attempt to merge it to any
of these alternates (Dhewm3, Morpheus, RBDoom3BFG, etc) and see if it's possible and what breakage results.

If any of these attempts are fruitful, then start tracking TDM SVN commits after the big v1.08 code merge and start adding them until
you reached parity with 2.02 (etc). Avoid most core infrastructure commits since almost none of that would apply to these other code
bases and just focus on AI, Scripting and mod specific changes.

Sure it would still be a bear to do but if anyone wanted to take advantage of the new contributions from the GPL community it's the best
bet other than paying Robert Beckebans, MH, or RaynorPat to join the project.

Hopefully nobody will scold me too much for such a speculation.
Please visit TDM's IndieDB site and help promote the mod:

http://www.indiedb.c...ds/the-dark-mod

(Yeah, shameless promotion... but traffic is traffic folks...)

#49 New Horizon

New Horizon

    Mod hero

  • Active Developer
  • PipPipPipPipPip
  • 13815 posts

Posted 16 June 2014 - 09:45 PM

Serpentine already has a good chunk of the work toward merging TDM and dhewm3 but it has been stalled since he disappeared.

#50 nbohr1more

nbohr1more

    Darkmod PR, Wordsmith

  • Development Role
  • PipPipPipPipPip
  • 8872 posts

Posted 16 June 2014 - 09:53 PM

From what I understand of his discussions, his approach was to diff the two code-bases and try to reconcile.
Every time new stuff was added to TDM he had to rework this whole process.

My suggestion makes the whole thing less risky and cumbersome.

Only focus on how TDM's SDK code differs from the game code and merge from there.

The added benefit is that the exercise should reveal what the current codebase has from TDM vs what it inherited.

Once you have an old TDM running on a new engine the barrier to upgrade it to a newer build is far lower than this gigantic
merge\reconcile process.

Yeah, it would be great if that Dhewm3 merge arrived one day though.

Edited by nbohr1more, 16 June 2014 - 09:56 PM.

Please visit TDM's IndieDB site and help promote the mod:

http://www.indiedb.c...ds/the-dark-mod

(Yeah, shameless promotion... but traffic is traffic folks...)





Also tagged with one or more of these keywords: idtech4, development, engine

1 user(s) are reading this topic

1 members, 0 guests, 0 anonymous users


    New Horizon