I'm working on a VR version - early alpha

[stgatilov]

Here is my tdm_update.log

It displays 2.06 in the main menu of the game, but only there. When I try opening it with VR mod installed and the game gives me the initialization error it's called 2.05:

According to the log, you should have proper 2.05 version.

I could suspect that probably you run 64-bit version, forgetting that 2.05 did not have one. But your log suggests that you did a clean download, so you should not even have the 64-bit executable.

 

I tried to update from same address starting with 2.07 installation.

I see version 2.05 both in menu and game console after update.

Maybe you should disable you antivirus and run the updater again with same settings.

[Bienie]

According to the log, you should have proper 2.05 version.

I could suspect that probably you run 64-bit version, forgetting that 2.05 did not have one. But your log suggests that you did a clean download, so you should not even have the 64-bit executable.

 

I tried to update from same address starting with 2.07 installation.

I see version 2.05 both in menu and game console after update.

Maybe you should disable you antivirus and run the updater again with same settings.

 

It's quite possible that I ran the x64 executable when it displayed 2.06. I tried again and now it's says 2.07...

 

I actually switched back to the original executable for 2.05 and I'm getting the same problem with "error during initialization" that I posted a screenshot of. That probably means that this problem is not related to the VR mod, but rather TDM 2.05. Any idea why the x64 executable works but not the x86? It works without a problem in my regular 2.07 install...

[stgatilov]

Perhaps you ran the wrong executable, which created config file.

Than you ran the proper 2.05 one, but it had to work with already existing config from another version.

Delete all executables which don't show 2.05 to you, and delete config file then.

And I think you can check console contents via logFile.

[Bienie]

Oh, ok I will try to delete all files except the tdm_update_tdmsync and tdm_mirrors and reinstall then, cause I just took my 2.07 and made a copy and updated. Maybe with a fresh install, and no x64 shenanigans the game will boot.

 

edit: Yup! That worked! Thanks stgatilov!

Edited May 17, 2019 by Bienie

[BuckleBean]

Just a quick note, I have a Valve Index set to arrive later this month. I'm very excited to take TDM for a spin with the new headset. Since it uses SteamVR, I expect it to work just as it did with the Vive & Rift. For what it's worth, I didn't have any luck using SteamVR input to rebind the keyboard/mouse controls to VR controller triggers/buttons. I may explore other solutions once I have the Index + controllers in hand.

[duzenko]

@cabalistic

Did you get any of BFG ParrallelJobList stuff working in your branch? I get into all kinds of memory errors when trying to apply it to models in frontend

[cabalistic]

I never tried - the frontend changes from BFG were too drastic. I did manage to parallelize some parts of the frontend, but it was a challenge to get the relevant data structures thread-safe. In the end, while it increased the frontend performance a bit, it made the backend run worse. Either the additional threads were stealing CPU time from the backend (which is potentially fixable), or the changes to the data structures were detrimental to the backend. I abandoned the effort, since at the time at least I didn't have any scenes were the frontend was a significant bottleneck compared to the backend.

[duzenko]

27 minutes ago, cabalistic said:

I never tried - the frontend changes from BFG were too drastic. I did manage to parallelize some parts of the frontend, but it was a challenge to get the relevant data structures thread-safe. In the end, while it increased the frontend performance a bit, it made the backend run worse. Either the additional threads were stealing CPU time from the backend (which is potentially fixable), or the changes to the data structures were detrimental to the backend. I abandoned the effort, since at the time at least I didn't have any scenes were the frontend was a significant bottleneck compared to the backend.

Right

With gl_BasePointer and persistent mapping I start seeing front end bottle-necking around AddModels.

So I'm trying to route it via ParallelJobList and getting all these errors.

I already switched tri allocator from vanilla D3 thread-unsafe static allocator and tried synchronizing numDrawSurfs but it's still crashing with corrupt memory and even looped linked lists...

[cabalistic]

Well, here's a relevant commit that shows what I did in my last attempt: https://github.com/fholger/thedarkmod/commit/87880984c894fd2dd68ba87e8bf34785ff5eaf1c

Note, this is almost two years old, things may have changed. I also used TBB at the time instead of porting BFG job queues for easier development. You'll see that I had to replace a number of the allocators because they are inherently not thread-safe, and use atomics at several places. These are not light changes to be made; even though it worked fine in my tests, they may not be safe. That's another reason why I abandoned the effort.

[duzenko]

5 minutes ago, cabalistic said:

Well, here's a relevant commit that shows what I did in my last attempt: https://github.com/fholger/thedarkmod/commit/87880984c894fd2dd68ba87e8bf34785ff5eaf1c

Note, this is almost two years old, things may have changed. I also used TBB at the time instead of porting BFG job queues for easier development. You'll see that I had to replace a number of the allocators because they are inherently not thread-safe, and use atomics at several places. These are not light changes to be made; even though it worked fine in my tests, they may not be safe. That's another reason why I abandoned the effort.

That commit has a rather different scope. It deals with lights and interactions while my focus is on adding ambient models

[cabalistic]

Yeah, but I have a feeling the challenges will be similar in some regards. Also, light interaction is one of the highest-scoring functions in the profiler from the frontend.

[duzenko]

On 10/3/2019 at 12:56 PM, cabalistic said:

Yeah, but I have a feeling the challenges will be similar in some regards. Also, light interaction is one of the highest-scoring functions in the profiler from the frontend.

You might want to check out the current trunk. Got about 15% fps increase with parrallel jobs and add-models.

Yeah, it's buggy right now, but with enough critical section hunting I hope to find and fix it.

[cabalistic]

At least yesterday evening trunk did not even compile for me...

[duzenko]

1 hour ago, cabalistic said:

At least yesterday evening trunk did not even compile for me...

Yikes, what was the error?

[cabalistic]

According to AppVeyor:

c:\projects\thedarkmod\typeinfo\main.cpp(203): error C2143: syntax error: missing ';' before '*' [C:\projects\thedarkmod\typeinfo.vcxproj]
c:\projects\thedarkmod\typeinfo\main.cpp(203): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int [C:\projects\thedarkmod\typeinfo.vcxproj]
c:\projects\thedarkmod\typeinfo\main.cpp(203): error C2065: 'MAX_THREADS': undeclared identifier [C:\projects\thedarkmod\typeinfo.vcxproj]
c:\projects\thedarkmod\typeinfo\main.cpp(206): error C2061: syntax error: identifier 'xthreadInfo' [C:\projects\thedarkmod\typeinfo.vcxproj]
c:\projects\thedarkmod\typeinfo\main.cpp(207): error C2065: 'info': undeclared identifier [C:\projects\thedarkmod\typeinfo.vcxproj]
c:\projects\thedarkmod\typeinfo\main.cpp(207): error C2182: 'Sys_DestroyThread': illegal use of type 'void' [C:\projects\thedarkmod\typeinfo.vcxproj]
c:\projects\thedarkmod\typeinfo\main.cpp(207): error C2365: 'Sys_DestroyThread': redefinition; previous definition was 'function' [C:\projects\thedarkmod\typeinfo.vcxproj]
  c:\projects\thedarkmod\idlib\sys\sys_threading.h(208): note: see declaration of 'Sys_DestroyThread'
c:\projects\thedarkmod\typeinfo\main.cpp(207): error C2448: 'Sys_DestroyThread': function-style initializer appears to be a function definition [C:\projects\thedarkmod\typeinfo.vcxproj]

I think there were additional errors, but this is the first one AppVeyor stumbles over. Also, on Linux:

[ 27%] Building CXX object CMakeFiles/TheDarkMod.dir/idlib/Thread.cpp.o
/home/appveyor/projects/thedarkmod/idlib/Thread.cpp: In static member function ‘static int idSysThread::ThreadProc(idSysThread*)’:
/home/appveyor/projects/thedarkmod/idlib/Thread.cpp:218:3: error: ‘_exit’ was not declared in this scope
   _exit( 0 );
   ^~~~~
/home/appveyor/projects/thedarkmod/idlib/Thread.cpp:218:3: note: suggested alternative: ‘_Exit’
   _exit( 0 );
   ^~~~~
   _Exit

 

[duzenko]

4 minutes ago, cabalistic said:

According to AppVeyor:

c:\projects\thedarkmod\typeinfo\main.cpp(203): error C2143: syntax error: missing ';' before '*' [C:\projects\thedarkmod\typeinfo.vcxproj]
c:\projects\thedarkmod\typeinfo\main.cpp(203): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int [C:\projects\thedarkmod\typeinfo.vcxproj]
c:\projects\thedarkmod\typeinfo\main.cpp(203): error C2065: 'MAX_THREADS': undeclared identifier [C:\projects\thedarkmod\typeinfo.vcxproj]
c:\projects\thedarkmod\typeinfo\main.cpp(206): error C2061: syntax error: identifier 'xthreadInfo' [C:\projects\thedarkmod\typeinfo.vcxproj]
c:\projects\thedarkmod\typeinfo\main.cpp(207): error C2065: 'info': undeclared identifier [C:\projects\thedarkmod\typeinfo.vcxproj]
c:\projects\thedarkmod\typeinfo\main.cpp(207): error C2182: 'Sys_DestroyThread': illegal use of type 'void' [C:\projects\thedarkmod\typeinfo.vcxproj]
c:\projects\thedarkmod\typeinfo\main.cpp(207): error C2365: 'Sys_DestroyThread': redefinition; previous definition was 'function' [C:\projects\thedarkmod\typeinfo.vcxproj]
  c:\projects\thedarkmod\idlib\sys\sys_threading.h(208): note: see declaration of 'Sys_DestroyThread'
c:\projects\thedarkmod\typeinfo\main.cpp(207): error C2448: 'Sys_DestroyThread': function-style initializer appears to be a function definition [C:\projects\thedarkmod\typeinfo.vcxproj]

I think there were additional errors, but this is the first one AppVeyor stumbles over. Also, on Linux:

[ 27%] Building CXX object CMakeFiles/TheDarkMod.dir/idlib/Thread.cpp.o
/home/appveyor/projects/thedarkmod/idlib/Thread.cpp: In static member function ‘static int idSysThread::ThreadProc(idSysThread*)’:
/home/appveyor/projects/thedarkmod/idlib/Thread.cpp:218:3: error: ‘_exit’ was not declared in this scope
   _exit( 0 );
   ^~~~~
/home/appveyor/projects/thedarkmod/idlib/Thread.cpp:218:3: note: suggested alternative: ‘_Exit’
   _exit( 0 );
   ^~~~~
   _Exit

 

Do we still need TypeInfo for anything? If so I would like to know if I'm missing on something useful.

The other error is something I can't grasp. Is it a cmake project? Do you build it with scons or??

[cabalistic]

Sorry, yes, I'm working on a CMake build system. It was just the first result from AppVeyor I found, wasn't intentional. Are you saying idlib/Thread.cpp should not be compiled on Linux? If so, it's my mistake. Otherwise it has nothing to do with CMake...

[duzenko]

20 minutes ago, cabalistic said:

Sorry, yes, I'm working on a CMake build system. It was just the first result from AppVeyor I found, wasn't intentional. Are you saying idlib/Thread.cpp should not be compiled on Linux? If so, it's my mistake. Otherwise it has nothing to do with CMake...

No, it should, but it seems to compile with 'default' scons command. I'm rather clueless when it comes to these command -line tools.

[duzenko]

https://stackoverflow.com/questions/3956636/c-on-linux-not-recognizing-commands-like-exit-and-printf

Where should we put all this include stuff? I might have missed it when importing from RB-BFG.

[cabalistic]

Include needs to go right in the file where it's needed.

[duzenko]

8 minutes ago, cabalistic said:

Include needs to go right in the file where it's needed.

But we have a precompiled header and a sys_includes so it might go there.

[cabalistic]

Imho, your code should not rely on the precompiled header to function correctly, it's just a compile speed improvement. In fact, not every file currently even includes the precompiled header - on MSVC it is auto-included in every file, but with SCons precompiled headers are not used and not auto-included, and that's one of the reasons why these kinds of missing headers regularly crop up in the Linux build

(My CMake build does use precompiled headers even with GCC, which significantly speeds up compile times and is more on par with the MSVC build.)

[duzenko]

1 hour ago, cabalistic said:

Imho, your code should not rely on the precompiled header to function correctly, it's just a compile speed improvement. In fact, not every file currently even includes the precompiled header - on MSVC it is auto-included in every file, but with SCons precompiled headers are not used and not auto-included, and that's one of the reasons why these kinds of missing headers regularly crop up in the Linux build

(My CMake build does use precompiled headers even with GCC, which significantly speeds up compile times and is more on par with the MSVC build.)

Could you offer any advice on how to make scons use precompiled headers as well? It takes forever to compile.

[cabalistic]

I got no clue. As far as I know, SCons does not directly support it, so you'd have to hack it in, somehow. I had to do the same in CMake (their native support for precompiled headers will only be available in an upcoming version), but I'm not knowledgable enough about SCons to repeat the feat. Nor would I want to - I'd rather replace SCons with CMake entirely.

[duzenko]

I started on multi draw calls again. I think you did that in VR branch already.

My problem: most surfaces have different model matrices. In my test only 10% of depth pass surfaces are in the world space. Simply calling MultiDrawElements does not handle that.

For that reason now I just only redirect the world space surfaces to multi draw.

What do you think is the best approach here? Uniform buffers for matrices? Would take quite an effort to generate it. Also, not sure of buffer upload overhead compared to more draw calls.

Grouping surfaces by space? How many draw calls would we save considering many entities have only one or few surfaces?