Jump to content


Photo

TDM Engine Development Page

idtech4 development engine

  • Please log in to reply
856 replies to this topic

#51 SteveL

SteveL

    Hero Coder

  • Active Developer
  • PipPipPipPip
  • 3669 posts

Posted 17 June 2014 - 11:15 AM

Comitted. EDIT: The source.


Just tried to compile it but it's complaining about omp.h being missing.

#52 Obsttorte

Obsttorte

    Scripting guru, Mapper

  • Active Developer
  • PipPipPipPipPip
  • 5609 posts

Posted 17 June 2014 - 01:02 PM

Download it. It's part of an dll called vccomp<version>.dll. Putting that into your darkmod directory should do the trick (unsure about that, as I did not need to do it).
FM's: Builder Roads, Old Habits, Old Habits Rebuild
WIP's: Several. Although after playing Thief 4 I really wanna make a city mission.
Mapping and Scripting: Apples and Peaches
Sculptris Models and Tutorials: Obsttortes Models
My wiki articles: Obstipedia
Let's Map TDM YouTube playlist: ObstlerTube
Texture Blending in DR: DR ASE Blend Exporter

End of shameless self promotion.

#53 nbohr1more

nbohr1more

    Darkmod PR, Wordsmith

  • Development Role
  • PipPipPipPipPip
  • 8872 posts

Posted 17 June 2014 - 01:29 PM

Obs may not have seen the problem if he compiled under Linux as OpenMP is included there as I recall?

(The current struggle is to get it working with Windows.)
Please visit TDM's IndieDB site and help promote the mod:

http://www.indiedb.c...ds/the-dark-mod

(Yeah, shameless promotion... but traffic is traffic folks...)

#54 Obsttorte

Obsttorte

    Scripting guru, Mapper

  • Active Developer
  • PipPipPipPipPip
  • 5609 posts

Posted 17 June 2014 - 01:31 PM

I've compiled under Windows ;)
  • nbohr1more likes this
FM's: Builder Roads, Old Habits, Old Habits Rebuild
WIP's: Several. Although after playing Thief 4 I really wanna make a city mission.
Mapping and Scripting: Apples and Peaches
Sculptris Models and Tutorials: Obsttortes Models
My wiki articles: Obstipedia
Let's Map TDM YouTube playlist: ObstlerTube
Texture Blending in DR: DR ASE Blend Exporter

End of shameless self promotion.

#55 revelator

revelator

    Member

  • Development Role
  • PipPip
  • 431 posts

Posted 18 June 2014 - 03:39 AM

Hello.

I been working on and off to get this thing working and some parts are indeed working just not giving any noticable performance improvements yet.
The last thing thats still keeping it from release is that the #pragma omp calls dont seem to work correctly on windows (patch originated from a linux user) and will hang the engine.
I do have the Intel compiler though and it should be possible to set it up to point out where omp would give the biggest benefit and also provide the nessesary code changes.

Ill post back if i get something going :).
  • AluminumHaste and nbohr1more like this

#56 nbohr1more

nbohr1more

    Darkmod PR, Wordsmith

  • Development Role
  • PipPipPipPipPip
  • 8872 posts

Posted 18 June 2014 - 07:29 AM

Thanks for your continued investigation on this Revelator!
Please visit TDM's IndieDB site and help promote the mod:

http://www.indiedb.c...ds/the-dark-mod

(Yeah, shameless promotion... but traffic is traffic folks...)

#57 taaaki

taaaki

    Forum Hoster

  • Root
  • 826 posts

Posted 20 June 2014 - 02:50 PM

I've got a bit of experience with omp (and TBB, and Boost Threads, and pthreads), so I'd like to have a look at the code / diffs.

More for the rest of us trying to compile and see the benefits: on Windows you'll need to enable OpenMP support in VS under the Engine project properties (C/C++ -> Language) and under Linux, the -fopenmp open needs to be set for GCC (the compilers ignore the omp pragmas otherwise).

I am the bat. The night is mine.


#58 grayman

grayman

    Master Builder, Coder

  • Active Developer
  • PipPipPipPipPip
  • 12564 posts

Posted 20 June 2014 - 10:24 PM

How long will it be before we can start building the SVN Linux version again?

#59 grayman

grayman

    Master Builder, Coder

  • Active Developer
  • PipPipPipPipPip
  • 12564 posts

Posted 21 June 2014 - 08:28 AM

Okay, I just got bit by the missing omp.h file. <_<

Can someone tell me exactly what I need to do to fix this?

And how did this not get included in the project in the first place, when it was first identified as needed?

#60 grayman

grayman

    Master Builder, Coder

  • Active Developer
  • PipPipPipPipPip
  • 12564 posts

Posted 21 June 2014 - 02:42 PM

BUMP!

I need to get the fix for this.

2.02 testing is waiting for it.

#61 grayman

grayman

    Master Builder, Coder

  • Active Developer
  • PipPipPipPipPip
  • 12564 posts

Posted 21 June 2014 - 02:46 PM

More for the rest of us trying to compile and see the benefits: on Windows you'll need to enable OpenMP support in VS under the Engine project properties (C/C++ -> Language)...


I added this for the Engine.

Building the DLL complains, so I also added it for the DLL. But the build is still complaining about not being able to find omp.h.

HELP!

#62 grayman

grayman

    Master Builder, Coder

  • Active Developer
  • PipPipPipPipPip
  • 12564 posts

Posted 21 June 2014 - 03:36 PM

I just got caught up with this thread.

We already have boost threads.

Why are we interested in OpenMP?

Is this simply to try to interest people in the community who work in open source areas?

I'm curious, because I've begun experimenting with boost threads to improve the loading time, and if we're moving to something completely different, then I need to wait for that to be proven and working before continuing the experiment.

#63 grayman

grayman

    Master Builder, Coder

  • Active Developer
  • PipPipPipPipPip
  • 12564 posts

Posted 21 June 2014 - 04:16 PM

Nothing but crickets here, so ...

After searching around, I found a comment that omp.h is not included in Visual Express. Thus the failures, since that's what I use to build.

I found an archive that would provide me with the missing pieces. After installing them, and making the project change suggested by taaaki, the build completed properly.

In the future, if someone finds that something missing is needed to support work that that person has committed to SVN, PLEASE ALERT THE REST OF US, and provide the missing parts and instructions on what project changes are needed, if they are. This will save us hours of lost time.

For anyone encountering this problem on Windows, here is the archive I found:

Attached File  OpenMP-Studio-Express.zip.txt   8.9KB   0 downloads

Now I have to go figure this out for Linux, since it's suffering the same problem.

#64 taaaki

taaaki

    Forum Hoster

  • Root
  • 826 posts

Posted 21 June 2014 - 04:16 PM

I added this for the Engine.

Building the DLL complains, so I also added it for the DLL. But the build is still complaining about not being able to find omp.h.

HELP!


My VS2010 has omp.h under C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\include\omp.h. Maybe check if it lives there in your Windows VM (might require the non-free version).

Boost Threads basically just provides a wrapper around a lower-level threading library like pthreads. OpenMP provides nice and simple way of defining parallelism using compiler pragmas. The compiler then inserts the relevant threading code (using pthreads or whatever the particular compiler uses - I guess MSVS uses the Windows threading libs). I don't see why both can't be used simultaneously. OpenMP is far simpler to implement and maintain in there areas where its use makes sense (data parallelism, certain task-based parallelism, etc.).

I am the bat. The night is mine.


#65 taaaki

taaaki

    Forum Hoster

  • Root
  • 826 posts

Posted 21 June 2014 - 04:17 PM

I actually reckon we just remove the current OpenMP changes. We should only be committing code to SVN that works. I'm happy to work with someone on this on the side or in a branch, but it shouldn't hit trunk until it's ready.

I am the bat. The night is mine.


#66 grayman

grayman

    Master Builder, Coder

  • Active Developer
  • PipPipPipPipPip
  • 12564 posts

Posted 21 June 2014 - 04:25 PM

I actually reckon we just remove the current OpenMP changes. We should only be committing code to SVN that works. I'm happy to work with someone on this on the side or in a branch, but it shouldn't hit trunk until it's ready.


The commits to SVN don't work, or they cause the builds on Express to not work?

SVN should only be receiving code changes that work. (In the case of the recent sikk changes, it was probably okay because they could be turned off at the Main menu.)

#67 taaaki

taaaki

    Forum Hoster

  • Root
  • 826 posts

Posted 21 June 2014 - 04:31 PM

The commits to SVN don't work, or they cause the builds on Express to not work?


It's not to do with the version of VS from what I can tell (although, I'm not actually sure if Express actually supports compiling with OpenMP). Even with the correct headers and compiler flags, the code in svn trunk is syntactically incorrect, so it won't compile. I'm just going to comment out the omp stuff and commit so that other work can continue (I'll also commit the updated SCons scripts for building with OpenMP support on Linux - will make it possible to toggle with or without).

I am the bat. The night is mine.


#68 New Horizon

New Horizon

    Mod hero

  • Active Developer
  • PipPipPipPipPip
  • 13815 posts

Posted 21 June 2014 - 06:50 PM

Yep, a branch would make the most sense since it's not fully working yet.

#69 nbohr1more

nbohr1more

    Darkmod PR, Wordsmith

  • Development Role
  • PipPipPipPipPip
  • 8872 posts

Posted 29 June 2014 - 05:46 PM

I have opened a bountysource tracker for porting BFG's GPU skinning to vanilla Doom 3:

https://www.bountyso...rt-gpu-skinning

This would be for raynorpat's morpheus branch which already has BFG's VBO and GLSL backend.

Whether this ever helps TDM, at least the campaign may help getting more activity going on the vanilla Doom 3 scene.
Please visit TDM's IndieDB site and help promote the mod:

http://www.indiedb.c...ds/the-dark-mod

(Yeah, shameless promotion... but traffic is traffic folks...)

#70 nbohr1more

nbohr1more

    Darkmod PR, Wordsmith

  • Development Role
  • PipPipPipPipPip
  • 8872 posts

Posted 07 July 2014 - 11:49 AM

So it seems revelator has a new repo to continue his previous work:

https://github.com/r...ator/Revelation

It looks like he added BFG's "constrain shadow volume to light volume and player view" optimizations...

He's also converting the built-in tools to wxwidgets similar to what Greebo is doing to Dark Radiant.
Please visit TDM's IndieDB site and help promote the mod:

http://www.indiedb.c...ds/the-dark-mod

(Yeah, shameless promotion... but traffic is traffic folks...)

#71 nbohr1more

nbohr1more

    Darkmod PR, Wordsmith

  • Development Role
  • PipPipPipPipPip
  • 8872 posts

Posted 20 July 2014 - 03:27 PM

I created a bountysource tracker for replacing the realtime script interpreter with a JIT implementation.
DanielGibson has recommended the use of LLVM for this feature:

https://www.bountyso...cript-compiling

Shall I open a local bugtracker for this too? Anyone familiar with LLVM?
Please visit TDM's IndieDB site and help promote the mod:

http://www.indiedb.c...ds/the-dark-mod

(Yeah, shameless promotion... but traffic is traffic folks...)

#72 taaaki

taaaki

    Forum Hoster

  • Root
  • 826 posts

Posted 20 July 2014 - 11:07 PM

Heh, that is actually an idea I had a while back (using LLVM for the script interpreter). I don't have any LLVM experience, but I do have a wee bit of (university) compiler development experience, so I was going to use this as a learning project. It's not on my current todo list though (which has been delayed trying to get a complete and correct SVN to Hg conversion going).

I am the bat. The night is mine.


#73 SteveL

SteveL

    Hero Coder

  • Active Developer
  • PipPipPipPip
  • 3669 posts

Posted 21 July 2014 - 12:52 AM

Has anyone done any profiling to see how much time tdm is spending in scripts and in handovers between the script interpreter and the game code? There's code in the animation scripts that I've been looking at recently that looks like it passes control back and forth several times per frame per ai, but I've no idea whether that has a noticeable effect or not. I don't know anything about llvm so can't contribute to that party of the conversation but I can help reduce the context switching if that turns out to be a burden. That's a big part of my day job: the main platform I develop on (Oracle) has an overhead for every context switch between pl/sql and sql, and in some apps it can really add up.

#74 taaaki

taaaki

    Forum Hoster

  • Root
  • 826 posts

Posted 21 July 2014 - 06:37 AM

My initial interest is in the script interpreter to LLVM idea was more for improved error-checking and warnings and not so much on the performance side. Like I said, it isn't on my current todo list, so I haven't given it too much thought. So yeah, don't expect anything from me on this topic until after I've finished off some other stuff and have some time to think about it properly (assuming someone else doesn't take up the task).

I am the bat. The night is mine.


#75 nbohr1more

nbohr1more

    Darkmod PR, Wordsmith

  • Development Role
  • PipPipPipPipPip
  • 8872 posts

Posted 21 July 2014 - 07:25 AM

Has anyone done any profiling to see how much time tdm is spending in scripts and in handovers between the script interpreter and the game code? There's code in the animation scripts that I've been looking at recently that looks like it passes control back and forth several times per frame per ai, but I've no idea whether that has a noticeable effect or not. I don't know anything about llvm so can't contribute to that party of the conversation but I can help reduce the context switching if that turns out to be a burden. That's a big part of my day job: the main platform I develop on (Oracle) has an overhead for every context switch between pl/sql and sql, and in some apps it can really add up.


You might glean some better insights from the Fabien Sanglard himself: fabiensanglard.net@gmail.com

If I interpret his code analysis here:

http://fabiensanglar...cripting_vm.php

correctly, such context changes definitely look problematic...
Please visit TDM's IndieDB site and help promote the mod:

http://www.indiedb.c...ds/the-dark-mod

(Yeah, shameless promotion... but traffic is traffic folks...)





Also tagged with one or more of these keywords: idtech4, development, engine

1 user(s) are reading this topic

1 members, 0 guests, 0 anonymous users


    New Horizon