TDM Engine Development Page

[duzenko]

Let's try it this way

 

The fboResolutuion refactoring happens in the guthub repo branch only.

The unrelated changes can go straight to svn trunk. Then if you want them in your pivate github branch - you pull them into the github *main* branch first and only then you merge master into your private branch.

Please no formatting-only changes.

 

When you have something for us to test in your private branch, me and probably @nbohr1more review it and I merge it to svn with a single commit. This way it can be reverted easily and does not get mixed up with other changes.

 

@nbohr1more, what are the things you usually test when I push something? I'd guess

• stencil shadows (hard and soft)
• shadow maps (hard and soft)
• multisampling
• ???

I'd also test the r_fboresolution in both < 1 and > 1 directions.

[revelator]

Sounds like a plan , looking over BFG atm to get a hint on how scaling works there.

I hope to have something soon, but BFG is rather differently setup than idtech4 so i might have to do a few workarounds.

[Bikerdude]

Nice work chaps

[revelator]

Umf, RBDoom3 BFG did not have an answer as such, fbo resizing is done in the image manager and its not configurable, still it made sense in a way.

Shadowmap scaling uses code similar to what i did with FB_Resize though he rolled his own version of R_GetModeInfo dedicated to that specific task.

 

With that in mind i would guess i need to look at FB_CopyRender since it seems that is where things interface with the renderer, image manager and whatnot ?.

[duzenko]

I would start with analyzing what t_fboResolution does in frontend and what frontend outputs get affected.

[HMart]

Hey guys just curious, have you discussed internally how to best support shadow maps for alpha mapped surfaces?

Edited October 25, 2018 by HMart

[duzenko]

Hey guys just curious, have you discussed internally how to best support shadow maps for alpha mapped surfaces?

What's there to discuss? It just works.

[HMart]

What's there to discuss? It just works.

 

I'm not seeing shadow mapped shadows for alpha materials in any mission besides the Judith test map for the volumetrics, i assume that is because past missions disable those shadows with the no_shadow material keyword, so my doubt is how will you force shadows on past maps. Will you like fhdoom just ignore the "no_shadow" keyword in alpha materials?

[Springheel]

Yes, that should be given some thought. I can easily imagine scenarios where ignoring "no_shadow" would cause undesired effects.

[duzenko]

Yes, that should be given some thought. I can easily imagine scenarios where ignoring "no_shadow" would cause undesired effects.

I don't get to decide that and I'm generally fine both ways

I would suggest a new 'no stencil shadows' material flag for future maps and the shadow transparency option could be "auto/forced"

[Springheel]

This would probably have to be set on the texture level, not the map. These are just a few issues that come to mind:

 

1. Graffiti decals, LOD skylines. Some use alpha transparency but no one wants shadows cast by them.

 

2. Moving foliage alpha textures. Forcing shadows on could potentially have a huge hit to FPS, but might be desirable for machines that can handle it (though might conflict with anything the mapper may have done to fake shadows already).

 

3. Transparent grate decals. Usually mappers simulate shadows with a light texture; forcing shadows on could cause weird conflicts.

 

4. Particles? Many of these also use alpha textures and obviously we don't want them casting shadows.

 

I can't really offer potential solutions since I have no idea how the new feature works.

[HMart]

This would probably have to be set on the texture level, not the map. There are just a few issues that come to mind:

 

1. Graffiti decals, LOD skylines. Some use alpha transparency but no one wants shadows cast by them.

 

2. Moving foliage alpha textures. Forcing shadows on could potentially have a huge hit to FPS, but might be desirable for machines that can handle it (though might conflict with anything the mapper may have done to fake shadows already).

 

3. Transparent grate decals. Usually mappers simulate shadows with a light texture; forcing shadows on could cause weird conflicts.

 

I can't really offer suggestions since I have no idea how it works.

 

That is exactly what i thought has well.

 

About decals this is not a silver bullet, but for example fhdoom shadow maps just ignore any material using textures from the decal folder to prevent them from casting shadows.

	if( coverage == MC_PERFORATED ) {

		//WARNING(johl): this is kind of hacky but is necessary to get the 
		//               original Doom3 materials to work at a decent perf.
		//               Decals don't cast a shadow, unfortunately there is no
		//               flag indicating whether a material is decal or not.
		//               Decals usually are alpha tested and have the 'noshadows'
		//               flag set, but 'noshadows' is ignored by fhDOOM on alphatested
		//               surfaces to enable alphatested shadows.
		//               Current solution: Just assume all materials starting with
		//               'textures/decals/' are decals and dont cast a shadows.
		static const char* const decalsPrefix = "textures/decals/";
		static const int decalsPrefixLen = strlen(decalsPrefix);

		if ( cullType != CT_TWO_SIDED && (strncmp( GetName(), decalsPrefix, decalsPrefixLen ) == 0)) {
			return false;
		}

		return true;
	}

Appart from decals, the sky and particles IMO all alpha materials should cast shadows by default, but of course mission makers could want to disable shadow maps, and this is where problems arise.

 

I'm certainly not thinking about everything right now, but I would suggest making a entity keyword (defined in the editor or def file) called no_shadowmap or something, why a entity keyword and not a material keyword like we currently have? It would certainly be easier to apply to many objects at the same time, yes true, but a material keyword, unless implemented in totally new materials, will mess with already made missions, where's a new entity keyword defined in the editor just affects the mission if the author wants it.

 

This of course do not solves the problem with past missions, forcing alpha shadows for them is not desirable, like Springheel explained and unless their makers update them, with the keyword above or similar, then is best to continue playing with alpha shadow off, that means we will have no alpha shadows at all in plenty of missions, and to me at least, that defeats the purpose of playing with them on, imo the only benefit of SM was alpha shadows, especially when in my case soft stencil shadows look very good and perform better than shadow mapping.

 

Afaik Is not a easy problem to solve and that is why I pushed for talking about it, final TDM 2.07 is still far away but is better to solve this sooner than later, just my two cents.

 

Edited October 27, 2018 by HMart

[peter_spy]

In the current build, using noshadows 1 either in the model, material, or light will disable alpha transparency shadow maps, so not sure what is the problem. It's already controllable.

[HMart]

In the current build, using noshadows 1 either in the model, material, or light will disable alpha transparency shadow maps, so not sure what is the problem. It's already controllable.

 

We know that Judith, the problem is if the team ignores the noshadows keyword , like fhdoom does, to force alpha shadows on past missions, in that case shadow maps will not be able to be turned off using noshadows, that is why i was suggesting a different keyword for shadow maps.

[peter_spy]

That would be a bad idea IMO, this is the kind of thing that should be authored every time, not automated. This is a new feature, and making things more complicated and difficult for current mappers, just to support legacy missions – I think it's bad approach. I understand striving for maximum compatibility, but it has to be reasonable. People generally understand that new features are for new stuff, and authors or community can always get back to old missions to implement that.

[Springheel]

So you're proposing we create new materials for all the textures that currently use alphamaps? And then skins for the appropriate models? They currently all have noshadows set in the material.

[nbohr1more]

To my knowledge, all alpha textures either have decal_macro or noshadows set in the material def. The only thing we should need to do is selectively edit material defs to remove those and allow shadows where we want them. This can be done gradually over time. There's no need to set any timetable other than having one example for 2.07.

[HMart]

To my knowledge, all alpha textures either have decal_macro or noshadows set in the material def. The only thing we should need to do is selectively edit material defs to remove those and allow shadows where we want them. This can be done gradually over time. There's no need to set any timetable other than having one example for 2.07.

 

Hum unless i'm not understanding what you are saying (is possible) i don't think that solves the problem of past missions faking shadows on alpha materials (don't think this is solvable unless the mission is edited) nor it solves the problem of people playing with stencil shadows on, if you play with stencil noshadows keyword should be there.

 

About a example Trees should be a obvious candidate.

 

 

Btw i thought of a little contrived way to solve the stencil shadow problem, make a copy of the alpha material you want to cast alpha shadows, remove the noshadows from that material, make a skin and on a script detect when the cvar r_shadows is 2 and if so change the necessary entities to the skin permitting alpha shadows...

Edited October 28, 2018 by HMart

[duzenko]

 

Hum unless i'm not understanding what you are saying (is possible) i don't think that solves the problem of past missions faking shadows on alpha materials (don't think this is solvable unless the mission is edited) nor it solves the problem of people playing with stencil shadows on, if you play with stencil noshadows keyword should be there.

 

About a example Trees should be a obvious candidate.

 

 

Btw i thought of a little contrived way to solve the stencil shadow problem, make a copy of the alpha material you want to cast alpha shadows, remove the noshadows from that material, make a skin and on a script detect when the cvar r_shadows is 2 and if so change the necessary entities to the skin permitting alpha shadows...

I agree with @nbohr1more except instead of removing the noshadows flag it'd better be replaced with the new nostencilshadows flag IMHO.

[duzenko]

On a side note, here's my today's svn change

 

 

Console history small change

When entering a new command and it exists in the history the old occurrences are now removed.
This way it won't allow duplicates any more.

History is now an idStrList instead of the fixed array of idEditField.
Why? Easier access to the string elements by index. Built-in AddUnique() function. One less index member and less maintenance code.

The in-game console history is now unlimited in elements count. When written out to disk, it's limited to 100 strings instead of 64 (no reason but makes it more 'human')

[peter_spy]

So you're proposing we create new materials for all the textures that currently use alphamaps? And then skins for the appropriate models? They currently all have noshadows set in the material.

Either delete noshadows keyword and start checking legacy maps if they work correctly, or inform mappers they can now use shadows for alpha maps. If anything, I'd refrain from introducing a keyword that duplicates existing functionality.

[Springheel]

To my knowledge, all alpha textures either have decal_macro or noshadows set in the material def. The only thing we should need to do is selectively edit material defs to remove those and allow shadows where we want them. This can be done gradually over time. There's no need to set any timetable other than having one example for 2.07.

 

That doesn't really solve the problem though. Consider the tree foliage textures used on many of our tree models. If we modify the texture to remove the noshadows flag, then suddenly scenes like the forest in NHAT, that were already suffering low FPS, could potentially suffer a huge performance decrease.

 

And as HMart mentions, if this is a feature that can be turned on and off from the menu, it gets even more complex, as the solution has to work for both cases. If we remove the "noshadows" from the foliage textures, and someone hasn't activated stencil shadows from the menu (if indeed that's how it works) then they'd see square blocks everywhere. That seems to rule out providing skin options and letting mappers choose.

 

Obviously (like any change to how things work) this is going to require some thought.

[duzenko]

 

That doesn't really solve the problem though. Consider the tree foliage textures used on many of our tree models. If we modify the texture to remove the noshadows flag, then suddenly scenes like the forest in NHAT, that were already suffering low FPS, could potentially suffer a huge performance decrease.

You'd need to prove that the NHAT forest is shadows limited first

[Springheel]

You'd need to prove that the NHAT forest is shadows limited first

 

I don't know what that means. It clearly suffers low FPS already. If you're saying suddenly turning on these new shadows won't have any performance impact, that would be an exciting development indeed. But even if performance magically becomes a non-issue, that still leaves the other issues raised above, like conflicts where mappers have already faked shadows using other methods like projected lights or decals.

[duzenko]

 

I don't know what that means. It clearly suffers low FPS already. If you're saying suddenly turning on these new shadows won't have any performance impact, that would be an exciting development indeed. But even if performance magically becomes a non-issue, that still leaves the other issues raised above, like conflicts where mappers have already faked shadows using other methods like projected lights or decals.

Most of the lights there are noshadows already. The few remaining lights are relatively small and don't intersect with much of the foliage.

Even we force the shadows on the foliage (which BTW I don't suggest to) I don't expect to see more than 5% change in draw calls and tri count.

 

None is more concerned with performance than me. I'm on IGP and I'm not afraid of this.

I wish you focused your performance concerns on missions beta testing - no offense intended.