Jump to content
The Dark Mod Forums

NOTICE: 32-bit executables deprecated


stgatilov

Do you play 32-bit or 64-bit build of TheDarkMod?  

25 members have voted

  1. 1. Do you play 32-bit or 64-bit build of TheDarkMod?

    • I play 64-bit build and think 32-bit build is useless.
      20
    • I play 64-bit build, but I think that 32-bit build is better in some regard.
      2
    • I play 32-bit build and cannot play 64-bit build at all.
      3


Recommended Posts

TheDarkMod offers 64-bit build since 2.06, and I can't recall any issues with with it since then.
I guess most of the people use 64-bit executable now. As far as I know, developers have long stopped testing 32-bit build 😁
Additionally, there is one recent gigantic FM which has too high memory requirements. As far as I remember, it cannot be dmapped with 32-bit build at all.

For the sake of clarity:

  • 64-bit executables are: TheDarkModx64.exe, thedarkmod.x64, tdm_update.linux64
  • 32-bit executables are: TheDarkMod.exe, thedarkmod.x86, tdm_update.linux

 

We plan to deprecate 32-bit builds after 2.08.

Here is exactly what would change:

  1. The website will offer 64-bit updater for Linux by default (now it offers 32-bit one).
  2. The updater will install only 64-bit game executables. 32-bit executables will not be packaged with the rest of the game.
  3. Developers will not test 32-bit build. Well, they don't test it now anyway 😁
  4. The 32-bit executables for the latest TDM release will be available as a separate download on website, without minimal-to-none testing.

Of course, we will not do anything to break 32-bit builds and will not remove any parts of it.
On developer side, we will still maintain 32-bit build, just with minimum effort. We will still ensure that it compiles, and probably test it from time to time.

 

I'd like to ask all the players with 32-bit OS to respond to this thread. You are the ones who will have to download the separate 32-bit executables to continue playing.

If you can play 64-bit executable but think 32-bit ones are useful in any regard, please post a comment too.

Link to comment
Share on other sites

In 2.06, the 64-bit executable had a performance deficit but as of 2.07 and later the two have parity.

The only thing that may be an outlier is Pentium 4 SIMD? Does any of the original id Software "P4" SIMD assembly exist in current 32-bit builds?

Should anyone on a Pentium 4 be playing TDM 2.08 or newer?

Please visit TDM's IndieDB site and help promote the mod:

 

http://www.indiedb.com/mods/the-dark-mod

 

(Yeah, shameless promotion... but traffic is traffic folks...)

Link to comment
Share on other sites

26 minutes ago, nbohr1more said:

In 2.06, the 64-bit executable had a performance deficit but as of 2.07 and later the two have parity.

More SIMD routines were reimplemented, so that's not surprising.

Quote

The only thing that may be an outlier is Pentium 4 SIMD? Does any of the original id Software "P4" SIMD assembly exist in current 32-bit builds?
Should anyone on a Pentium 4 be playing TDM 2.08 or newer?

I don't know what is "Pentium 4 SIMD".
The SIMD code contains a lot of assembly written by ID for Doom 3. They are enabled only in 32-bit build, there is no way to make them work in 64-bit build. But I suppose all performance-heavy functions out of them have already been reimplemented for 64-bit build.

Pentium 4 was the top CPU when Doom 3 came out. I think nobody has it now.

Link to comment
Share on other sites

Here is some background on how Doom 3 was tailored to the P4:

https://fabiensanglard.net/doom3_documentation/37729-293751.pdf

https://fabiensanglard.net/doom3_documentation/37727-293749.pdf

https://fabiensanglard.net/doom3_documentation/37728-293750.pdf

https://fabiensanglard.net/doom3_documentation/37726-293748.pdf

https://fabiensanglard.net/doom3_documentation/37730-293752.pdf

 

Not to impugn the esteemed J.M.P. van Waveren but I am fairly certain that a few coins landed in his pockets when he prominently mentioned

Pentium 4 in those technical reviews...

Please visit TDM's IndieDB site and help promote the mod:

 

http://www.indiedb.com/mods/the-dark-mod

 

(Yeah, shameless promotion... but traffic is traffic folks...)

Link to comment
Share on other sites

8 minutes ago, nbohr1more said:

Here is some background on how Doom 3 was tailored to the P4:

All of them are about SIMD code.

The reasons he mentions P4 are 1) It was the most recent Intel CPU at the moment of Doom 3 release, and 2) It was the first CPU to support SSE2 instructions.
Before P4, there was only SSE, which is very limited: it only supports working on floats. SSE2 extended SIMD to doubles and integers of all sizes, making it much more versatile.
Today SSE2 is required by TDM, and guaranteed in x64 mode.

Link to comment
Share on other sites

What does it entails to change a code base to 64bits? Just change the SSE and SIMD code?

About the pool, my case is not there, I play the 32bits version but I can also play the 64bits version just fine, I just chose to play the 32bits version because I assumed it to be the most stable version, but if is not tested anymore, and there's no performance diference than I will play with 64bits one, thanks for the info stgatilov. 

Link to comment
Share on other sites

Does really anyone play TDM on 32-bit-only hardware in 2020?

If not i would recommend dropping 32 bit support completely, so devs don't have to avoid breaking it every time they work on the engine.
Code is not only an asset but also a burden. It has to be maintained. Less code is easier to maintain.

Link to comment
Share on other sites

2 hours ago, Abusimplea said:

Does really anyone play TDM on 32-bit-only hardware in 2020?

The main concern is not people with 32-bit only hardware, but people who have installed 32-bit OS for some reason. This is pretty easy to do even today.

Quote

If not i would recommend dropping 32 bit support completely, so devs don't have to avoid breaking it every time they work on the engine.

If 32-bit build gets broken while 64-bit is not, this is most likely a sign of bad and non-portable code, which should be fixed anyway, and which most likely can hurt in 64-bit build too.
Perhaps other developers have different habits, but I can say for myself that I have tested 32-bit build for something like 10 times since 2.07, and it seems to work properly without much effort.
 

  • Like 1
Link to comment
Share on other sites

On 5/24/2020 at 9:40 PM, HMart said:

What does it entails to change a code base to 64bits? Just change the SSE and SIMD code?

It was a pretty hard work to make engine work in 64-bits, because ID wrote code in very platform-specific manner, they never expected it to become 64-bit.
However, all that work was finished by 2.06, and no more effort was spent on it since then.

Quote

About the pool, my case is not there, I play the 32bits version but I can also play the 64bits version just fine, I just chose to play the 32bits version because I assumed it to be the most stable version, but if is not tested anymore, and there's no performance diference than I will play with 64bits one, thanks for the info stgatilov. 

It would be great if people do independent performance testing. To be honest, I did not check it for a long time already, and one guy (nbohr1more) is not enough to draw a final conclusion.
I believe the performance difference is not very high though.

Link to comment
Share on other sites

5 hours ago, stgatilov said:

The main concern is not people with 32-bit only hardware, but people who have installed 32-bit OS for some reason. This is pretty easy to do even today.
 

I guess you could pose the same question then: Is there really anyone who plays TDM on a 32-bit OS nowadays? :)

I'm sure there is. 1 or 2. AFAIC, you could easily drop 32-bit support, but, of course your decision.

Edited by chakkman
  • Like 1
Link to comment
Share on other sites

I'm not a coder so my opinion is arguably biased, but I think 32-bit is terrible.

Old, faulty, limiting. It's simply not worth it anymore, it was great back in the days because, heh, 16-bit, anybody remember that?

Everybody should use 64-bit and be forced to use it...Although I'm a bit of a hypocrite saying that because I hate Windows 10, yet I use it, yet I want Windows 7 back but I'm paranoid of hackers especially since I got trembled by them 9 times, 3 were successful.

Link to comment
Share on other sites

Worth noting that Microsoft is getting ready to kill off support for 32-bit Windows, and most Linux flavors killed native support for 32-bit installs years ago. Not Debian, which is what I use (but not for that reason).

 

My vote is to ditch 32-bit support if it makes things easier for the team. I used to play TDM on a high-end 3GHz Pentium 4, and the experience was less than satisfactory anyway. Only the simplest in-door missions ran well.

 

As time goes by, people will want to use larger textures and better maps/models, and the memory cap of 32-bit becomes more and more of a problem in this scenario.

Link to comment
Share on other sites

Even tho I do agree that TDM should go 64bits exclusively I most say this:

Using 64bits software doesn't mean 32 bits goes away, why, first because it is neither old nor faulty, afaik is just a size of a memory address (an integer), and second because even tho idtech 4 is or was 32bits software, it still uses 8bits and 16bits wide data, and TDM 64bits still uses 8,16 and 32 bit wide data.  

Going 64bit, does make a game faster, just because the team has the option to use bigger memory address numbers and so, access more RAM, but this is only important if your game is memory starved.  Other thing that 64bits gives is the option to have more precision on some CPU/GPU calculations, that does mean better looking effects but also mean heavier calculations and so slower calculations, talking in global terms, is that what you need for your game? Do you lose much by doing the 32bits calculations instead? This is what game/engine devs need to think about.

Having the option to do more, also means you can be more wasteful about memory usage/optimization, the game can end bloated very fast and slower to many people using low end hardware. 

Edited by HMart
  • Like 1
Link to comment
Share on other sites

So many misconceptions here, I guess I should say something.

13 hours ago, HMart said:

Using 64bits software doesn't mean 32 bits goes away, why, first because it is neither old nor faulty

Yes, 32-bit is perfectly OK in general.

Quote

Going 64bit, does make a game faster, just because the team has the option to use bigger memory address numbers and so, access more RAM, but this is only important if your game is memory starved. 

There is limitation of 2GB of memory in 32-bit mode. And that's a hard limit: a process cannot use more than 2 GB even if you have 16GB RAM physically installed and 32GB pagefile for swap. When trying to allocate more, process will simply crash on 32-bit platform.

If your workload fits well into this limit, then there is no harm from it. If it does not, then it becomes an unavoidable blocker. If you have a very big map, you simply cannot dmap it on 32-bit, but if your map is small/medium size, it does not matter for you if executable is 32-bit or 64-bit.

Quote

even tho idtech 4 is or was 32bits software, it still uses 8bits and 16bits wide data, and TDM 64bits still uses 8,16 and 32 bit wide data.  

Speaking of variables in code and 64-bit mode switch, some of them stay the same size, but some of then switch from 32-bit to 64-bit.

All the pointers inevitably become two times larger, and all sizes in STL types do the same. Also, 64-bit process has more overhead for heap allocations. It means that the process consumes more memory in 64-bit, and consequently can work slower since the cache size does not change. You can probably see increased memory consumption yourself: take a big brushy map, and dmap it both in 32-bit and in 64-bit modes, looking at memory consumption in process explorer.

While this point is not very strong for native applications and games, it is a major problem for managed languages like Java, where pointers are everywhere. For instance, modern JVM still uses 32-bit pointers internally, as long as you don't request for more than 32 GB of heap memory. They say it is 30% faster 😂

Quote

Other thing that 64bits gives is the option to have more precision on some CPU/GPU calculations, that does mean better looking effects but also mean heavier calculations and so slower calculations, talking in global terms, is that what you need for your game? Do you lose much by doing the 32bits calculations instead? This is what game/engine devs need to think about.

64-bit mode does not offer any new capabilities except for more-than-2GB memory.
There are many other differences between 32-bit and 64-bit, but they are all about performance.

64-bit mode does not offer better precision. Floating point numbers are 32-bit and 64-bit both on 32-bit and 64-bit, working with same performance. In fact, 32-bit mode also offers 80-bit floating point numbers. Moreover, before something like TDM 2.06, this awful 80-bit arithmetic was used everywhere for intermediate values when computing complicated expressions. We got many precision problems when we disabled it, and we are still fixing them (e.g. there is a bunch of such fixes for dmap in upcoming 2.08). Now 80-bit floats are disabled both for 32-bit and 64-bit builds of TDM.
So it turns out that 32-bit mode offers more floating point precision, although these 80-bit numbers cannot be used universally in Visual Studio anyway.
(assembly geeks can also notice that 32-bit mode without 80-bit arithmetic has a tiny bit of overhead for passing floats into functions, due to old calling convention)

Speaking of integers, both 32-bit and 64-bit modes offer integers of size up to 64-bit. The 64-bit integers are emulated on 32-bit platform and are pretty slow, but TDM never uses them. 64-bit GCC also offers builtin 128-bit integer type, but Visual Studio does not, and so TDM does not use them. Anyway, I can hardly imagine what we could use them for 😯
(assembly geeks can also notice that using 32-bit integers in 64-bit mode occasionally have a bit of overhead, due to some additional extending instructions)

And of course bitness of executable does not affect GPU 😁

 

The main performance advantage of 64-bit mode is having 2x more registers in CPU: both GP registers and SSE/AVX registers.
When compiler goes out of registers in a tight loop, it has to "spill" into memory. This memory is surely in L1 cache, but L1 cache accesses are still slower than registers, plus more dependencies between instructions.
In TDM, there are many computationally heavy routines done in SSE/AVX (aka SIMD), which we had to rewrite for 64-bit mode. Because the original implementation by ID was written in inline assembly, and 1) you cannot use 32-bit assembly on 64-bit build, and 2) VC does not allow inline assembly at all in 64-bit build. So there are two completely different implementations of SIMD routines now: the first one (by ID) is used in 32-bit mode, and the second one (by us) is used in 64-bit mode. As far as I remember, the second one is often pretty low on registers, so if we start using it in 32-bit mode, it will work a bit slower. Some of the new routines written by us are already used in 32-bit mode, e.g. AVX code.

 

Quote

Having the option to do more, also means you can be more wasteful about memory usage/optimization, the game can end bloated very fast and slower to many people using low end hardware. 

Quite unlikely I would say.
Only that 32-bit build will start working slower than 64-bit build. People with low-end machines should use 64-bit build, so it's not a big problem.

 

As you see, performance difference is a very complicated question.
Out of all I wrote, I think the most difference comes from:

  1. 64-bit build wastes more memory (pointers, heap overhead)
  2. 64-bit build has more registers
  3. 64-bit and 32-bit builds have different SIMD implementations

To be honest, no idea which build wins now.

  • Like 3
Link to comment
Share on other sites

10 hours ago, stgatilov said:

There is limitation of 2GB of memory in 32-bit mode. And that's a hard limit: a process cannot use more than 2 GB even if you have 16GB RAM physically installed and 32GB pagefile for swap. When trying to allocate more, process will simply crash on 32-bit platform.

Interesting read, but this is not correct. There is the /LARGEADRESSAWARE flag with which you can allocate upto about 3.5 GB on a 32-bit process.

https://docs.microsoft.com/de-de/cpp/build/reference/largeaddressaware-handle-large-addresses?view=vs-2019

And we actually use the flag on TDM.

Link to comment
Share on other sites

26 minutes ago, STiFU said:

Interesting read, but this is not correct. There is the /LARGEADRESSAWARE flag with which you can allocate upto about 3.5 GB on a 32-bit process.

This is just a minor detail.
Yes, you can enable PAE and use 32 GB on a 32-bit Windows server OS, as long as you are ready to live with segmented addressing (like in 16-bit times).

Link to comment
Share on other sites

On 5/31/2020 at 6:45 AM, stgatilov said:

So many misconceptions here, I guess I should say something.

Yes, 32-bit is perfectly OK in general.

There is limitation of 2GB of memory in 32-bit mode. And that's a hard limit: a process cannot use more than 2 GB even if you have 16GB RAM physically installed and 32GB pagefile for swap. When trying to allocate more, process will simply crash on 32-bit platform.

If your workload fits well into this limit, then there is no harm from it. If it does not, then it becomes an unavoidable blocker. If you have a very big map, you simply cannot dmap it on 32-bit, but if your map is small/medium size, it does not matter for you if executable is 32-bit or 64-bit.

Speaking of variables in code and 64-bit mode switch, some of them stay the same size, but some of then switch from 32-bit to 64-bit.

All the pointers inevitably become two times larger, and all sizes in STL types do the same. Also, 64-bit process has more overhead for heap allocations. It means that the process consumes more memory in 64-bit, and consequently can work slower since the cache size does not change. You can probably see increased memory consumption yourself: take a big brushy map, and dmap it both in 32-bit and in 64-bit modes, looking at memory consumption in process explorer.

While this point is not very strong for native applications and games, it is a major problem for managed languages like Java, where pointers are everywhere. For instance, modern JVM still uses 32-bit pointers internally, as long as you don't request for more than 32 GB of heap memory. They say it is 30% faster 😂

64-bit mode does not offer any new capabilities except for more-than-2GB memory.
There are many other differences between 32-bit and 64-bit, but they are all about performance.

64-bit mode does not offer better precision. Floating point numbers are 32-bit and 64-bit both on 32-bit and 64-bit, working with same performance. In fact, 32-bit mode also offers 80-bit floating point numbers. Moreover, before something like TDM 2.06, this awful 80-bit arithmetic was used everywhere for intermediate values when computing complicated expressions. We got many precision problems when we disabled it, and we are still fixing them (e.g. there is a bunch of such fixes for dmap in upcoming 2.08). Now 80-bit floats are disabled both for 32-bit and 64-bit builds of TDM.
So it turns out that 32-bit mode offers more floating point precision, although these 80-bit numbers cannot be used universally in Visual Studio anyway.
(assembly geeks can also notice that 32-bit mode without 80-bit arithmetic has a tiny bit of overhead for passing floats into functions, due to old calling convention)

Speaking of integers, both 32-bit and 64-bit modes offer integers of size up to 64-bit. The 64-bit integers are emulated on 32-bit platform and are pretty slow, but TDM never uses them. 64-bit GCC also offers builtin 128-bit integer type, but Visual Studio does not, and so TDM does not use them. Anyway, I can hardly imagine what we could use them for 😯
(assembly geeks can also notice that using 32-bit integers in 64-bit mode occasionally have a bit of overhead, due to some additional extending instructions)

And of course bitness of executable does not affect GPU 😁

 

The main performance advantage of 64-bit mode is having 2x more registers in CPU: both GP registers and SSE/AVX registers.
When compiler goes out of registers in a tight loop, it has to "spill" into memory. This memory is surely in L1 cache, but L1 cache accesses are still slower than registers, plus more dependencies between instructions.
In TDM, there are many computationally heavy routines done in SSE/AVX (aka SIMD), which we had to rewrite for 64-bit mode. Because the original implementation by ID was written in inline assembly, and 1) you cannot use 32-bit assembly on 64-bit build, and 2) VC does not allow inline assembly at all in 64-bit build. So there are two completely different implementations of SIMD routines now: the first one (by ID) is used in 32-bit mode, and the second one (by us) is used in 64-bit mode. As far as I remember, the second one is often pretty low on registers, so if we start using it in 32-bit mode, it will work a bit slower. Some of the new routines written by us are already used in 32-bit mode, e.g. AVX code.

 

Quite unlikely I would say.
Only that 32-bit build will start working slower than 64-bit build. People with low-end machines should use 64-bit build, so it's not a big problem.

 

As you see, performance difference is a very complicated question.
Out of all I wrote, I think the most difference comes from:

  1. 64-bit build wastes more memory (pointers, heap overhead)
  2. 64-bit build has more registers
  3. 64-bit and 32-bit builds have different SIMD implementations

To be honest, no idea which build wins now.

Hey thanks for the deep explanation, I'm still learning this stuff, so is no surprise to me that I add some wrong or incomplete ideas but i'm glad I said it, because it made you correct me and that will just improve my knowledge. :)

  • Like 1
Link to comment
Share on other sites

1 hour ago, HMart said:

Hey thanks for the deep explanation, I'm still learning this stuff, so is no surprise to me that I add some wrong or incomplete ideas but i'm glad I said it, because it made you correct me and that will just improve my knowledge. :)

Don't worry, it's a very common misunderstanding. We talk frequently of "32 bit integers" or "64 bit floats" so people naturally assume that when we talk about "32 bit code" or "64 bit code" we mean the same thing, but in fact there is no connection between the size of memory addresses (which is what "64 bit code" refers to) and the ability to choose arbitrary sizes for integer or floating point data.

8, 16, 32 and 64 bit integers were already available in 32 bit code and remain available in 64 bit code, as do 32 bit and 64 bit floating point values, and often extended data types like the 80 bit "long double" floating points that stgatilov mentioned. The choice of which to use is determined by the size of the values that need to be represented, along with performance considerations and/or the amount of memory available.

  • Like 1
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Recent Status Updates

    • nbohr1more

      TDM 15th Anniversary Contest is now active! Please declare your participation: https://forums.thedarkmod.com/index.php?/topic/22413-the-dark-mod-15th-anniversary-contest-entry-thread/
       
      · 0 replies
    • JackFarmer

      @TheUnbeholden
      You cannot receive PMs. Could you please be so kind and check your mailbox if it is full (or maybe you switched off the function)?
      · 1 reply
    • OrbWeaver

      I like the new frob highlight but it would nice if it was less "flickery" while moving over objects (especially barred metal doors).
      · 4 replies
    • nbohr1more

      Please vote in the 15th Anniversary Contest Theme Poll
       
      · 0 replies
    • Ansome

      Well then, it's been about a week since I released my first FM and I must say that I was very pleasantly surprised by its reception. I had expected half as much interest in my short little FM as I received and even less when it came to positive feedback, but I am glad that the aspects of my mission that I put the most heart into were often the most appreciated. It was also delightful to read plenty of honest criticism and helpful feedback, as I've already been given plenty of useful pointers on improving my brushwork, level design, and gameplay difficulty.
      I've gotten back into the groove of chipping away at my reading and game list, as well as the endless FM catalogue here, but I may very well try my hand at the 15th anniversary contest should it materialize. That is assuming my eyes are ready for a few more months of Dark Radiant's bright interface while burning the midnight oil, of course!
      · 4 replies
×
×
  • Create New...