Doom3 GPL Source Release has landed

[stgatilov]

Wow!

I've been told about D3 going open source today... And I see that it's for real!

Now I fell morally obligated to join TDM for the upcoming Christmas:)

 

Staying tuned with other Doom3 forks is definitely a problem. Maybe we can avoid doing heavy changes to the engine at first time? Though it'd be almost the same as working with closed engine...

Anyway, setting up the infrastructure is first priority, isn't it? And refactoring the dirtiest hacks is the second. I added some of them myself:)

[Serpentine]

The fork projects (i.e not random hacks) have yet to really get going - tho there is a lot of hacking going on at the moment. iodoom3 seems to be waiting for a large patch coming in from someone 'important', before it gets off the ground formally. OpenD3 seems to just be a collection of patches from random git forks, not very interesting. Quite a few dependencies need removing/optional changes made, cmake seems to be gaining use - however the makelists are still a bit basic and need to mature a bit, however that will be a good step up (not that scons is all that bad). A few people are working on x64/amd64 support, which will make life easier, as well as an openbsd port which seems to be weeding out a bunch of non-portable code. Since no one is really working towards features at this stage, waiting and watching (while other work is going on in current TDM) will let us pick a good place to start off without duplicating too much work.

 

At the moment priority is still kicking 1.07 out the front door, after that it's cleanup and infrastructure time.

[etdarkb]

So, lets assume in the begin that users still need pak00{0,1,2,3,4}.pk4 from the original doom3 what modification we should make to the doom3 engine to improve TDM?

[Tirek]

If I may add my 2 cents;

IMHO, given the focus of TDM as a toolkit, to be used mostly by a community of unpaid fans, I think the most worthwhile improvements (apart from the obvious performance tweaks that access to the renderer enables) would have to be those that make life easier for the mappers and asset creators and that increase the flexibility of the engine... To explain a bit better: I think it would make sense to aspire towards "streamlining" the mapping process, in the sense that there are as few technological obstacles between the ideas and concepts that the mapper starts with and the realization of the same in the engine.

 

For instance, visibility determination that relies completely on mapper-placed portals requires a considerable amount of time to be spent on optimizing a map and even so, the mapper is still restricted to making fairly "boxed up" areas. Also, hard limits regarding brush and entity counts, "leaking" maps... These are the sort of obstacles I had in mind. Unfortunately, most of them seem to stem from the basic approach the engine uses to represent level geometry, AFAIK.

 

In any case, such wide ranging changes are hard to implement. I'd guess harder than, say, adding SSAO, or even replacing stencil shadows with shadow mapping and such.

 

Synchronizing various community efforts will probably prove to be the hardest part :/ I doubt TDM team actually has the manpower to manage it's own fork of idTech4.

[New Horizon]

Our primary goal right now will be getting TDM 1.07 out the door.

 

With the renewed interest in the D3 source, it may end up helping us if there are any optimizations / advances made to the engine that would work with TDM.

 

Sikkpin has already made one such change over on D3world where he has split ambient lighting off into it's own .vfp file. Currently the interaction.vfp file covers all lighting, if I understand correctly. In any case, there was an ambient light fix that added depth shading...given D3's current setup, the calculations for this fix had to be done on every light, which equaled a rather big performance hit. If the fixes were performed within the renderer, and only to ambient lights within the map...we would see an increase in performance and lower end PC's should then be able to experience the higher end visual options in our menu.

 

I think community efforts to restore Carmack's reverse will likely make it into every D3 Source project out there, and I highly doubt creative is going to go after anyone for it.

 

Other potential improvements might be tighter integration between Dark Radiant and the D3 renderer. It would make previewing your map much easier if it could tap into the map preview already available in D3Edit.

 

In any case, there are more possibilities than ever...but we'll have to choose those we can realistically do ourselves and keep an eye out for help outside TDM for anything that may strengthen the project.

[Tels]

Wow! I've been told about D3 going open source today... And I see that it's for real! Now I fell morally obligated to join TDM for the upcoming Christmas:) Staying tuned with other Doom3 forks is definitely a problem. Maybe we can avoid doing heavy changes to the engine at first time? Though it'd be almost the same as working with closed engine... Anyway, setting up the infrastructure is first priority, isn't it? And refactoring the dirtiest hacks is the second. I added some of them myself:)

 

Yeah, that is the plan. First release v1.07, then set up our infrastructure/read the code/compile it and then only look at modifying it. The future looks bright (if we can find the programmers to do the work, tho

[demagogue]

Other potential improvements might be tighter integration between Dark Radiant and the D3 renderer. It would make previewing your map much easier if it could tap into the map preview already available in D3Edit.

 

We could also set up DR to set EAX & location settings for individual portal leafs ... and get transparent windows to occlude sound, maybe get more realistic sound propagation (not through walls) if we're lucky. Heh, funny how I think of all the sound improvements first.

[rebb]

In any case, there was an ambient light fix that added depth shading...given D3's current setup, the calculations for this fix had to be done on every light, which equaled a rather big performance hit. If the fixes were performed within the renderer, and only to ambient lights within the map...we would see an increase in performance and lower end PC's should then be able to experience the higher end visual options in our menu.

 

Judging from the source it's very easy to (finally) make this fix - so much that one wonders why id left their ambientLight.vfp disabled in the first place, maybe it had to do with compatibility of the hardware back then - or the old "meh, good enough" mindset.

Instead of switching around the normalization cubemap which it currently does, they could have just loaded the dedicated vfp. Oh well.

[OrbWeaver]

maybe it had to do with compatibility of the hardware back then. Instead of switching around the normalization cubemap which it currently does, they could have just loaded the dedicated vfp. Oh well.

 

Yeah, perhaps it was very slow to switch between vertex/fragment programs on certain hardware which was current at the time Doom 3 was released.

[rebb]

It's possible - the weird thing is that the particular function actually does ARB Program binding everytime it's invoked. But maybe OpenGL ends up skipping unnecessary rebinding of the same program internally.

[zergrush]

So if TDM is really going standalone doesn't that imply a future name change? Like The Dark Game?

[jaxa]

It'll always be a modification of something, sooo...

[Springheel]

So if TDM is really going standalone doesn't that imply a future name change? Like The Dark Game?

 

The chances of a standalone TDM are not high, but they're higher than us changing the name.

[pusianka]

ehm... one short question. Will TDM use now more then 1 core of the CPU?

[New Horizon]

ehm... one short question. Will TDM use now more then 1 core of the CPU?

 

I don't think that would be something we'll tackle ourselves, but if the different D3 source projects are successful in doing so it may be something we could look into integrating. It's not an easy task though, so I wouldn't expect anything for quite some time.

[nbohr1more]

Doom 3 already has some threading but it's not currently setup to specify core affinity but rather relies on the OS to decide where threads go. For TDM a big stumbling block for taking advantage of threading is that the SDK interfaces do not allow anything parallel. Everything must be reduced to 1 thread before it is fed back to the executable. Once TDM is integrated into the Doom 3 source as one executable, it may automatically utilize some of the limited threading in the engine. Then, as NH mentioned, the open source community will likely add some improved multi-core code. The biggest performance improvement will be moving various rendering functions to the GPU and using more modern (and efficient) GPU techniques. Once the CPU isn't doing as much heavy lifting for the renderer it'll have more capacity to deal with AI.

[grayman]

Once the CPU isn't doing as much heavy lifting for the renderer it'll have more capacity to deal with AI.

 

Mmmmmm ... more capacity to deal with AI.

[New Horizon]

Mmmmmm ... more capacity to deal with AI.

 

[Shadowhide]

so what team will be doing after 1.07 release

[New Horizon]

so what team will be doing after 1.07 release

 

We will be discussing what to do. lol

[lost_soul]

If I had millions of dollars, I would hire the devs to work on the mod full time. Also, I tried to compile Doom 3 with that patch someone posted above that would use GLSL to render shadows. It failed somewhere in draw_common.cpp, which is what I edited. I probably did it wrong. I just took out the section mentioning the workaround and stuck the code from the post at Inside3d in there.

Edited November 26, 2011 by lost_soul

[zergrush]

One question, will this eventually help TDM run a little more smoothly? To be honest, my machine is a little outdated and I get choppy frame rates from time to time.

[New Horizon]

One question, will this eventually help TDM run a little more smoothly? To be honest, my machine is a little outdated and I get choppy frame rates from time to time.

 

That's a difficult question to answer with absolute certainty. We can certainly be hopeful that there will be a few optimizations that will smooth things out.

[Tels]

One question, will this eventually help TDM run a little more smoothly? To be honest, my machine is a little outdated and I get choppy frame rates from time to time.

 

Frankly, the best bet here is to buy new hardware. You could of course also join the team and work with us - but honestly, I don't think TDM will get "much more faster" - there will be a few optimizations we *can* do, but most of them require better hardware (multiple cores, a strong GPU, more memory) anyway.

 

Currently there is easily a factor of 1000 in performance between a medium PC from this year, and anything you bought like 3..5 years ago - and we cannot magically make our code 10..100 times faster - even 2 times faster is almost impossible except in rare circumstances. That means a newer PC will still laugh at the workload while your old machine will still struggle. So, new hardware is the easiest thing - it doesn't even have top of the line or high-end - just anything modern will help.

[zergrush]

I know, I know, I was just wondering how far could optimization go. I was planning to get a better machine in the near future either way. I was frankly more concerned on whether or not TDM will go standalone, although Springhell already thinned that possibility out. It's just that I would really love to see TDM reaching over to a broader public without being depedant on a Doom 3 instalation.