Jump to content


Photo

Threading discussion / testing with Cabalistic.


  • Please log in to reply
12 replies to this topic

#1 New Horizon

New Horizon

    Mod hero

  • Active Developer
  • PipPipPipPipPip
  • 13769 posts

Posted 25 July 2017 - 12:37 PM

Just a place to discuss threading implementation in the engine....concerns / benefits!


  • Bikerdude, jaxa, Anderson and 1 other like this

#2 jaxa

jaxa

    Advanced Member

  • Member
  • PipPipPip
  • 1235 posts

Posted 25 July 2017 - 04:24 PM

How is it that Cabalistic comes in and makes an instant threaded performance improvement (not sure how much performance gain, but he says "and it helped quite a bit") after years of talk and not a lot of change (that I could see)? And Doom 3 BFG: Edition has been GPL'd since 2012.

 

How far can multithreading go for TDM? Large open/outdoor scenes and adding additional AI seems to be the most FPS intensive things mappers can do. Can they be paralleled more?

 

More gamers than ever before are about to have 8 cores and 16 threads now that AMD's Ryzen 7 is out. Some of you may even lay down the cash for Threadripper with up to 32 threads, or the Intel Core i9 equivalents. Is it time for more threading, and will any of you be getting these CPUs?


  • Anderson likes this

#3 Anderson

Anderson

    Advanced Member

  • Member
  • PipPipPip
  • 788 posts

Posted 25 July 2017 - 04:27 PM

Don't know about 32 threads but my potato could use another thread used as it helped with any other game around. Weak Celerons need any push they can squeeze.

Imagine the changes you'll see!


 "I really perceive that vanity about which most men merely prate — the vanity of the human or temporal life. I live continually in a reverie of the future. I have no faith in human perfectibility. I think that human exertion will have no appreciable effect upon humanity. Man is now only more active — not more happy — nor more wise, than he was 6000 years ago. The result will never vary — and to suppose that it will, is to suppose that the foregone man has lived in vain — that the foregone time is but the rudiment of the future — that the myriads who have perished have not been upon equal footing with ourselves — nor are we with our posterity. I cannot agree to lose sight of man the individual, in man the mass."...

 

 

- 2 July 1844 letter to James Russell Lowell from Edgar Allan Poe.

 


#4 jaxa

jaxa

    Advanced Member

  • Member
  • PipPipPip
  • 1235 posts

Posted 25 July 2017 - 04:47 PM

Intel has enabled hyperthreading on some (all?) of its new Pentiums:

 

http://www.tomshardw...tium,33386.html

http://www.pcworld.c...oost-sales.html

https://www.digitalt...yper-threading/


  • Anderson likes this

#5 Judith

Judith

    Advanced Member

  • Member
  • PipPipPip
  • 596 posts

Posted 26 July 2017 - 05:52 AM

More gamers than ever before are about to have 8 cores and 16 threads now that AMD's Ryzen 7 is out. Some of you may even lay down the cash for Threadripper with up to 32 threads, or the Intel Core i9 equivalents. Is it time for more threading, and will any of you be getting these CPUs?

 

No and no. At least for games, it's mostly unnecessary, as they still use 4 cores, and that won’t change much until next generation of consoles comes along. It’s still hard to say when that happens, as Sony execs say there’s PS5 in the works, while MS execs state that “console generations are over” and it’s time to switch to incremental updates, as with mobile hardware. I’d be switching to some better hardware mostly due to better Mental Ray performance, and video encoding. Since that’s a hobby, I can wait long after prices drop. If gamers are really moving to 8-16 cores, we'll see that in Steam hardawre poll sooner or later, but I don't think so.

 

Edit: link to steam hardware CPU trends: http://store.steampo.../hwsurvey/cpus/

 

It looks like only recently users with quad core CPUs made more than 50% of the total.


Edited by Judith, 26 July 2017 - 10:09 AM.

  • Anderson likes this

#6 cabalistic

cabalistic

    Member

  • Member
  • PipPip
  • 80 posts

Posted 26 July 2017 - 10:43 AM

Hm, I'm not so sure games won't start using more threads soon. The way that modern parallelization is usally done is task-based, where a set of repeating code is divided into tasks which can run in parallel. These tasks are then executed on worker threads, and it doesn't really matter to the app how many worker threads you have. On a console you may have two workers, and on a PC you can easily spawn more if there are more cores available. So consoles aren't really the limiting factor here, it's more adopting the technique.

 

@jaxa: Unfortunately, adopting the Doom 3 BFG threading model is not that simple and would be quite a significant effort. There have been a lot of changes under the hood to accomodate it. I only did a fairly simple cut between frontend and backend, and even that took me quite a while. Not to mention that it's entirely unclear if it really is stable. After all, we have at least one report with issues, so I'd really appreciate more people trying it out :)


  • Anderson likes this

#7 jaxa

jaxa

    Advanced Member

  • Member
  • PipPipPip
  • 1235 posts

Posted 26 July 2017 - 02:15 PM

and that won’t change much until next generation of consoles comes along. It’s still hard to say when that happens, as Sony execs say there’s PS5 in the works, while MS execs state that “console generations are over”

 

PS4 and Xbox One and their refreshed versions, PS4 Pro and Xbox One X, already have eight cores, not four. Up to seven of which are usable by games (6 at launch). Only one thread per core though. They are also more like PCs than ever before because they use AMD x86 chips rather than PowerPC (Xbox 360) or Cell (PS3).

 

http://www.tomshardw...cked,30662.html

http://gamingbolt.co...ut-cloud-gaming

 

Vulkan is better at spreading work across more CPU cores AFAIK.

 

As for the Steam stats, 6-8 core will tick up (more than they already have) eventually. Ryzen sales have apparently helped AMD perform well in Q2 although the GPU shortage is also a factor. Some (loaded) gamers may be holding out for Intel's Core i9 CPUs or AMD's Threadripper.

 

The first Ryzen 8-cores became available on March 2nd. If the Steam Windows playerbase grew a bit, then the increase of 8-core CPUs from 0.24% in Feb to 0.47% today is basically a doubling, and it should continue. Ryzen 7 prices have also declined a bit from launch prices. Even if 8-core is a minimum of about $300, that leaves the Ryzen 5 1600X and Ryzen 5 1600 with 6 cores from $200-$230.

 

Obviously, it should be assumed that most PC gamers are using 2-4 cores. But the genie is out of the bottle and we will see more PC users switching to 6-8 cores, if not 16-18. Intel's Coffee Lake will include a mainstream-priced 6-core chip. The only place I don't see change is in laptops. Although you can stuff a desktop chip and a cooling system in a laptop, laptop CPU/APUs still top out at quad-core and there are no plans that I see to change that with Raven Ridge/Ryzen Mobile or the successor Gray Hawk, which may or may not be 7nm.



#8 Judith

Judith

    Advanced Member

  • Member
  • PipPipPip
  • 596 posts

Posted 26 July 2017 - 05:13 PM

But there's no reason to switch to 8 cores or more if games aren’t using them. Also remember that changing CPUs is not like buying a new GPU. You need to buy a good motherboard, fast compatible RAM, stable PSU, good cooling, etc. That will cost you a lot, so people make that purchase once a 6 years or even more, to have a solid foundation for GPUs they buy every 2-3 years or so.


  • CarltonTroisi likes this

#9 jaxa

jaxa

    Advanced Member

  • Member
  • PipPipPip
  • 1235 posts

Posted 26 July 2017 - 07:50 PM

PS4 and Xbox One games can use at least 6 threads. Developers would be foolish not to since the Jaguar cores are comparatively slow. Game development is moving towards more parallelism. Newer games are utilizing more than 4 cores:

 

https://www.reddit.c..._gaming_cpu_is/

https://www.reddit.c...re_than_4cores/

(now you can tell I'm being lazy)

 

If you have a tight budget, spending an extra $100 on a better graphics card is the right move, but we are chipping through the quad-core ceiling and it's clear that the 2017-18 CPU releases by AMD and Intel are going to put the hardware in the hands of more people. Game developers will still ensure most dual and quad-core users can play games, but games can scale to use the available hardware. There is a chicken and egg problem here, and "cheap" hex/octo/deca (compare the i7-6950X to the i9-7900X) cores being sold even if users don't necessarily need them today is a possible solution.

 

And there is another reason for someone to have more than 4 cores... if that person is a streamer.



#10 Judith

Judith

    Advanced Member

  • Member
  • PipPipPip
  • 596 posts

Posted 27 July 2017 - 03:02 AM

Yup, but I think this is still more like a tech-enthusiast perspective. It’s 2017, and, according to Steam poll, the biggest group of gamers still use video cards with 1-2 gigs of VRAM. My guess is that they won’t jump on a new CPU platform so readily and easily.



#11 cabalistic

cabalistic

    Member

  • Member
  • PipPip
  • 80 posts

Posted 30 July 2017 - 03:19 AM

So, some bad news. This weekend, I experimented with further parallelizing some parts of the frontend renderer (specifically, finding lights and adding interactions), which were especially prominent in the profiler. However, it made no difference at all, at least not in places where extra performance would have been needed. It appears that in cases where the frontend takes long, then the backend also takes long, so cutting down calculation times on the frontend didn't help anything, because the backend then becomes the bottleneck.

 

So this means further threading does not currently help, as the primary bottleneck is now actually the rendering. So to get more performance, I think there is now little choice but to port the vertexcache changes from Doom3 BFG, which would be a major undertaking...


  • jaxa and Anderson like this

#12 duzenko

duzenko

    Advanced Member

  • Active Developer
  • PipPipPip
  • 570 posts

Posted 30 July 2017 - 03:23 AM

Try Afterburned and see if you're on 100% gpu load.

If you are, then no CPU improvements will help.



#13 cabalistic

cabalistic

    Member

  • Member
  • PipPip
  • 80 posts

Posted 30 July 2017 - 03:47 AM

No, it's still entirely CPU bound, my GPU is bored to death (says SteamVR). But it is now bound in the backend, where parallelization is very difficult.






0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users