TDM Engine Development Page

[SteveL]

The depth duffer is an indespensible tool for rendering. He could improve SSAO, do tile-based occlusion evaluation, etc. Almost all of the magic in Sikkmod comes from a hack where he re-renders the whole scene to grab the depth buffer. Here it seems SteveL is rendering the native one at full speed (no re-render hack).

 

That's right. The grey panel is displaying the original early depth pass that idTech4 does before any shaders are applied. The video demonstrates that it can be made available to all shaders without any post-processing. I made the patch material above an alpha blend but that was only so that the patch itself didn't show up in the depth render. _currentDepth can be used even by opaque solid geometry shaders on the very first draw pass.

 

final version if I understood correctly that is

Yep that looks good. I'll use it instead of stripping out the test code from my version. Only change will be we can use GL_NONE instead of GL_BACK.

 

You did most of the work for this to be fair, so def pat yourself on the back

[revelator]

Yep that looks good. I'll use it instead of stripping out the test code from my version. Only change will be we can use GL_NONE instead of GL_BACK.

 

Sounds good, yeah that one had me a bit baffled.

 

And thanks but i could not have completed it without someone like you who could do the shader part for testing .

 

So now onto hacking apart sikkmods SSAO kinda wish sikkpin was here, i think he would love getting his hands dirty now that the code for getting the depthimage is complete.

[revelator]

Checked again, and small whoops it is indeed GL_BACK not GL_NONE GL_NONE was for FBO rendering but the depthbuffer copy needs GL_BACK.

Should actually have thrown an exception there but may not have been fatal so it passed anyway, probably a good idea with a GL_CheckError in that function just to make sure.

[Tels]

Steve, good work!

 

I know it is probably to early to optimize the ARB (and maybe even pointless?), but it caught my eye:

 

!!ARBvp1.0
OPTION ARB_position_invariant;
MOV result.texcoord, vertex.texcoord;
END

!!ARBfp1.0
PARAM clipPlanes = { 0.4, 350.0 };
TEMP  dpth, col, rng;
TEX   dpth, fragment.texcoord, texture[0], 2D;
ADD   rng.x, clipPlanes.y, -clipPlanes.x;

MUL   col.x, dpth.x, rng.x;					 // col.x = dpth.x * rng.x
ADD   col.x, -col.x, clipPlanes.x;		   // col.x = -col.x + clipPlanes.x

 

There is a MAD instruction, so you could probably write (untested)

 

MAD   col.x, -dpth.x, rng.x, clipPlanes.x;	 // col.x = (dpth.x * rng.x) * -1 + clipPlanes.x

 

If i'm correct, the end formula is:

 

2 * clipPlanes.y * -clipPlanes.x * 1 / ( (clipPlanes.y * -clipPlanes.x) * dpth.x * -1 + clipPlanes.x + clipPlanes.y )

 

It also looks like you might not even need the temp variable rng, but I haven't traced it completely

 

I found this link helpful: http://www.renderguild.com/gpuguide.pdf

 

Anyway, since these shaders are quite hard to read (except maybe for people like me, who grew up writing asm instructions for 8-bit CPUs manually... ) a few more comments would be good.

 

But nevertheless, great work!

[SteveL]

Checked again, and small whoops it is indeed GL_BACK not GL_NONE GL_NONE was for FBO rendering but the depthbuffer copy needs GL_BACK. Should actually have thrown an exception there but may not have been fatal so it passed anyway, probably a good idea with a GL_CheckError in that function just to make sure.

Ok. It worked last night with my drivers for whatever reason, but I notice GL_NONE isn't documented as valid for the function. GL_BACK it is.

 

How does Sikkpin's SSAO work? There are quite a few methods in that rendering book I was reading. Does it sample around each texel to see whether the surrounding geometry is closer than the texel, and shade if so? Or something more sophisticated?

 

But nevertheless, great work!

 

Thanks. Another thing I could do is feed through all the rgb values instead of just using x all the time and save a couple of MOVs at the end, but NB that shader won't become part of production code. It was purely a one-off to test whether the capture was working. And re the lack of comments, it was 2am A good comment to have used would have been the linearizing formula it implements:

# (2.0 * zNear) / (zFar + zNear - depth * (zFar - zNear));

 

EDIT: now I see you suggested that already...

Edited October 9, 2014 by SteveL

[Tels]

Thanks. Another thing I could do is feed through all the rgb values instead of just using x all the time and save a couple of MOVs at the end, but NB that shader won't become part of production code. It was purely a one-off to test whether the capture was working. And re the lack of comments, it was 2am A good comment to have used would have been the linearizing formula it implements:

# (2.0 * zNear) / (zFar + zNear - depth * (zFar - zNear));

 

EDIT: now I see you suggested that already...

 

No worries

 

One question: The zNear and zFar values are hard-coded. How does that work in praxis, are they later to be computed dynamically somehow? Or are these part of the engine, e.g. each engine needs different values?

[grayman]

Is it because of AI in general, or because of how idTech 4 is structured ?

 

Both, I think. Human behavior is a tough nut to crack. Always has been, always will be. The best we can do is simulate what we need for the situations the game presents.

 

Nearly all (I think) of id's AI behavior was done in scripts. Those scripts (and I assume id's AI structure) were ported to C++, and that formed the basis for TDM's AI. This would have been a lot quicker than designing a structure from scratch. A lot of TDM-specific human behavior was layered in on top of that afterward, and that work continues today.

 

Do I wish the structure was different? Maybe. Maybe not. My early work with AI (40 years ago) was centered around a goal-oriented structure. Bring me the red book, Mr. Robot, and do everything you need to do to accomplish that goal. TDM's AI aren't so much goal-oriented as they are "Here's the next micro-thing you should do": "Walk from A to B." "Say something if you encounter a friend." "React if a dead body tells you it's lying on the floor."

 

id's needs for monsters were simple: attack the player and kill him.

 

TDM's needs are way more complex. I think the frustration comes from "In the context of the original AI structure, how do I teach the AI to do this new human behavior, while at the same time not clobbering any of the things they've already been taught to do?"

 

I've toyed with the idea of changing the structure. I know greebo had his own ideas as well. But there's never been any time to refactor. We don't get paid to do this. We do what's fun to do. For me, if it ain't falling-down broken, it doesn't need fixing.

 

TDM's AI do a reasonable job of pretending they know what they're doing. That speaks volumes for the original port from id's code, and the work that's been done these past 7 years. Even though there are moments of sheer frustration behind the curtain, it's gratifying when Robert the guard finally learns what you're trying to teach him, and he comes on stage and performs and the audience gives him a standing ovation.

 

None of this work is a breeze. If it was, I'd prolly go do something more challenging.

[Springheel]

he depth duffer is an indespensible tool for rendering. He could improve SSAO, do tile-based occlusion evaluation, etc. Almost all of the magic in Sikkmod comes from a hack where he re-renders the whole scene to grab the depth buffer. Here it seems SteveL is rendering the native one at full speed (no re-render hack).

 

Very cool! Good work guys; can't wait to see how this pays off.

[revelator]

Sikkpins SSAO used a vile hack to get the depth image aside from that it works quite ok but breaks when colliding with some shader effects like the heathaze and skybox shaders.

 

The effect caused outlines to be drawn on alpha materials so whenever there was one of said effects you could see the outlined box.

It looked quite ghastly whenever that happened, but in the places where it worked it looked bloody amazing

[revelator]

Heres sikkpins SSAO shader from sikkmod.

 

!!ARBvp1.0
OPTION ARB_position_invariant;
# input:
#--------------------------------
# texcoord[0] = TEX0 texcoords
#
# local[0]  = sample radius, darkening, amount
MOV  result.texcoord[0], vertex.texcoord[0];
MUL  result.texcoord[1], program.local[0].x, { 1.0, 1.0, 2.0, 4.0 };
MOV  result.texcoord[2], program.local[0].y;
MOV  result.texcoord[3], program.local[0].z;
END
#=====================================================================================
!!ARBfp1.0
OPTION ARB_precision_hint_nicest;
# texture 0 = _depth
#
# env[0] = 1.0 to _currentRender conversion
# env[1] = fragment.position to 0.0 - 1.0 conversion
OUTPUT  oColor  = result.color;
ATTRIB fPos = fragment.position;
ATTRIB TC  = fragment.texcoord[0];
ATTRIB Radius = fragment.texcoord[1];
ATTRIB Bias = fragment.texcoord[2];
ATTRIB Amount = fragment.texcoord[3];
PARAM  nonPoT = program.env[0];
PARAM  invRes = program.env[1];
PARAM dcode24 = { 1.0, 0.003921568627450980392156862745098, 0.000015259021896696421759365224689097 };
PARAM  farClip = 8192.0;
PARAM  inv8 = 0.125;
PARAM  HQScale = 0.5;
PARAM  DefVal = 0.55;
PARAM  randUV = 0.25; # 0.25 = 4x4; 0.0625 = 16x16; 0.015625 = 64x64
# pre-normalized w/ other necessary math done
PARAM vec1 = {  0.01262953713852306359863762957346,  0.01262953713852306359863762957346,  0.01262953713852306359863762957346 };
PARAM vec2 = { -0.02525907427704612719727525914692, -0.02525907427704612719727525914692, -0.02525907427704612719727525914692 };
PARAM vec3 = { -0.03788861141556919079591288872038, -0.03788861141556919079591288872038,  0.03788861141556919079591288872038 };
PARAM vec4 = { -0.05051814855409225439455051829384,  0.05051814855409225439455051829384, -0.05051814855409225439455051829384 };
PARAM vec5 = { -0.06314768569261531799318814786729,  0.06314768569261531799318814786729,  0.06314768569261531799318814786729 };
PARAM vec6 = {  0.07577722283113838159182577744076, -0.07577722283113838159182577744076, -0.07577722283113838159182577744076 };
PARAM vec7 = {  0.08840675996966144519046340701422, -0.08840675996966144519046340701422,  0.08840675996966144519046340701422 };
PARAM vec8 = {  0.10103629710818450878910103658768,  0.10103629710818450878910103658768, -0.10103629710818450878910103658768 };
TEMP depthEnc, depth, random, ssao, uv;
TEMP  radius, dist, aspect, diff, res;
TEMP SampleScale, distScaled, DepthRangeScale, DepthTestSoftness, RangeIsInvalid;
TEMP R0, R1, R2, R3, R4, R5, R6, R7, R8;
TEMP S0, S1, S2, S3, S4, S5, S6, S7, S8;
TEMP  D1, D2, D3, D4;

# calculate the screen texcoord in the 0.0 to 1.0 range
# MUL  R0, fPos, invRes;
# scale by the screen non-power-of-two-adjust
# MUL  R0, R0, nonPoT;
# scale by the screen non-power-of-two-adjust
# MUL  R0, TC, nonPoT;
MOV  uv, 0.0;
MUL  uv.xy, fPos, invRes;
# calculate screen width/height
RCP  res.x, invRes.x;
RCP  res.y, invRes.y;
# calculate aspect ratio
MOV  aspect.x, 1.0;
MUL  aspect.y, res.x, invRes.y;
# load the depth render
TEX  depthEnc, uv, texture[0], 2D;
DP3  depth, depthEnc, dcode24;
# range conversions
MUL  depth.z, depth, farClip;
# MAX  depth.w, depth.z, 64.0;
# RCP  depth.w, depth.w;
MUL_SAT R1.x, depth.z, 0.0078125;   # make area smaller if distance less than 128 units
MAD  R1.y, depth.z, 0.001953125, 1.0; # make area bigger if distance more than 512 units
MUL  R1.x, R1.x, R1.y;
MUL  SampleScale.xyz, R1.x, Radius;
RCP  R1.x, SampleScale.z;
MUL  R1.x, farClip, R1.x;
MUL  DepthRangeScale, R1.x, 0.85;
# convert from units into SS units
RCP  R1.x, depth.z;
MUL  SampleScale.xy, SampleScale, R1.x;
MUL  SampleScale.z, SampleScale.z, 0.000244140625;
MUL  SampleScale.xy, SampleScale, aspect;
RCP  R1.x, SampleScale.z;
MUL  DepthTestSoftness, 32.0, R1.x;
# load random vector map
MUL  R0.xy, fPos, randUV;
TEX  random, R0, texture[2], 2D;
MAD  random, random, 2.0, -1.0;
DP3  random.w, random, random;
RSQ  random.w, random.w;
MUL  random.xyz, random, random.w;
###=================================================
# calculate offsets
DP3  R1.w, random, vec1;
MUL  R1.xyz, random, R1.w;
MAD  R1.xyz, R1, -2.0, vec1;
DP3  R2.w, random, vec2;
MUL  R2.xyz, random, R2.w;
MAD  R2.xyz, R2, -2.0, vec2;
DP3  R3.w, random, vec3;
MUL  R3.xyz, random, R3.w;
MAD  R3.xyz, R3, -2.0, vec3;
DP3  R4.w, random, vec4;
MUL  R4.xyz, random, R4.w;
MAD  R4.xyz, R4, -2.0, vec4;
DP3  R5.w, random, vec5;
MUL  R5.xyz, random, R5.w;
MAD  R5.xyz, R5, -2.0, vec5;
DP3  R6.w, random, vec6;
MUL  R6.xyz, random, R6.w;
MAD  R6.xyz, R6, -2.0, vec6;
DP3  R7.w, random, vec7;
MUL  R7.xyz, random, R7.w;
MAD  R7.xyz, R7, -2.0, vec7;
DP3  R8.w, random, vec8;
MUL  R8.xyz, random, R8.w;
MAD  R8.xyz, R8, -2.0, vec8;
MUL  S1.xyz, R1, SampleScale;
MUL  S2.xyz, R2, SampleScale;
MUL  S3.xyz, R3, SampleScale;
MUL  S4.xyz, R4, SampleScale;
MUL  S5.xyz, R5, SampleScale;
MUL  S6.xyz, R6, SampleScale;
MUL  S7.xyz, R7, SampleScale;
MUL  S8.xyz, R8, SampleScale;
# snap texcoord to pixel center
MUL  S1.xy, S1, res;
FLR  S1.xy, S1;
MUL  S2.xy, S2, res;
FLR  S2.xy, S2;
MUL  S3.xy, S3, res;
FLR  S3.xy, S3;
MUL  S4.xy, S4, res;
FLR  S4.xy, S4;
MUL  S5.xy, S5, res;
FLR  S5.xy, S5;
MUL  S6.xy, S6, res;
FLR  S6.xy, S6;
MUL  S7.xy, S7, res;
FLR  S7.xy, S7;
MUL  S8.xy, S8, res;
FLR  S8.xy, S8;
# do first sample set
MAD_SAT R1.xy, S1, invRes, uv;
MAD_SAT R2.xy, S2, invRes, uv;
MAD_SAT R3.xy, S3, invRes, uv;
MAD_SAT R4.xy, S4, invRes, uv;
MAD_SAT R5.xy, S5, invRes, uv;
MAD_SAT R6.xy, S6, invRes, uv;
MAD_SAT R7.xy, S7, invRes, uv;
MAD_SAT R8.xy, S8, invRes, uv;
# sample depth texture
TEX  R1, R1, texture[0], 2D;
TEX  R2, R2, texture[0], 2D;
TEX  R3, R3, texture[0], 2D;
TEX  R4, R4, texture[0], 2D;
TEX  R5, R5, texture[0], 2D;
TEX  R6, R6, texture[0], 2D;
TEX  R7, R7, texture[0], 2D;
TEX  R8, R8, texture[0], 2D;
# decode 24-bit depth
DP3  D1.x, R1, dcode24;
DP3  D1.y, R2, dcode24;
DP3  D1.z, R3, dcode24;
DP3  D1.w, R4, dcode24;
DP3  D2.x, R5, dcode24;
DP3  D2.y, R6, dcode24;
DP3  D2.z, R7, dcode24;
DP3  D2.w, R8, dcode24;
ADD  D1.x, D1.x, S1.z;
ADD  D1.y, D1.y, S2.z;
ADD  D1.z, D1.z, S3.z;
ADD  D1.w, D1.w, S4.z;
ADD  D2.x, D2.x, S5.z;
ADD  D2.y, D2.y, S6.z;
ADD  D2.z, D2.z, S7.z;
ADD  D2.w, D2.w, S8.z;
# do second sample set
MUL  S1.xyz, S1, HQScale;
MUL  S2.xyz, S2, HQScale;
MUL  S3.xyz, S3, HQScale;
MUL  S4.xyz, S4, HQScale;
MUL  S5.xyz, S5, HQScale;
MUL  S6.xyz, S6, HQScale;
MUL  S7.xyz, S7, HQScale;
MUL  S8.xyz, S8, HQScale;
MAD_SAT R1.xy, S1, invRes, uv;
MAD_SAT R2.xy, S2, invRes, uv;
MAD_SAT R3.xy, S3, invRes, uv;
MAD_SAT R4.xy, S4, invRes, uv;
MAD_SAT R5.xy, S5, invRes, uv;
MAD_SAT R6.xy, S6, invRes, uv;
MAD_SAT R7.xy, S7, invRes, uv;
MAD_SAT R8.xy, S8, invRes, uv;
# sample depth texture
TEX  R1, R1, texture[0], 2D;
TEX  R2, R2, texture[0], 2D;
TEX  R3, R3, texture[0], 2D;
TEX  R4, R4, texture[0], 2D;
TEX  R5, R5, texture[0], 2D;
TEX  R6, R6, texture[0], 2D;
TEX  R7, R7, texture[0], 2D;
TEX  R8, R8, texture[0], 2D;
# decode 24-bit depth
DP3  D3.x, R1, dcode24;
DP3  D3.y, R2, dcode24;
DP3  D3.z, R3, dcode24;
DP3  D3.w, R4, dcode24;
DP3  D4.x, R5, dcode24;
DP3  D4.y, R6, dcode24;
DP3  D4.z, R7, dcode24;
DP3  D4.w, R8, dcode24;
ADD  D3.x, D3.x, S1.z;
ADD  D3.y, D3.y, S2.z;
ADD  D3.z, D3.z, S3.z;
ADD  D3.w, D3.w, S4.z;
ADD  D4.x, D4.x, S5.z;
ADD  D4.y, D4.y, S6.z;
ADD  D4.z, D4.z, S7.z;
ADD  D4.w, D4.w, S8.z;
###=============================================
# calculate occlusion
ADD  dist, depth.x, -D1;
MUL  distScaled, dist, DepthRangeScale;
ABS_SAT R1, distScaled;
MOV_SAT R2, distScaled;
ADD  R1, R1, R2;
MUL  RangeIsInvalid, R1, 0.5;
MUL_SAT R1, -dist, DepthTestSoftness;
LRP  ssao, RangeIsInvalid, DefVal, R1;
ADD  dist, depth.x, -D2;
MUL  distScaled, dist, DepthRangeScale;
ABS_SAT R1, distScaled;
MOV_SAT R2, distScaled;
ADD  R1, R1, R2;
MUL  RangeIsInvalid, R1, 0.5;
MUL_SAT R1, -dist, DepthTestSoftness;
LRP  R1, RangeIsInvalid, DefVal, R1;
ADD  ssao, ssao, R1;
ADD  dist, depth.x, -D3;
MUL  distScaled, dist, DepthRangeScale;
ABS_SAT R1, distScaled;
MOV_SAT R2, distScaled;
ADD  R1, R1, R2;
MUL  RangeIsInvalid, R1, 0.5;
MUL_SAT R1, -dist, DepthTestSoftness;
LRP  R1, RangeIsInvalid, DefVal, R1;
ADD  ssao, ssao, R1;
ADD  dist, depth.x, -D4;
MUL  distScaled, dist, DepthRangeScale;
ABS_SAT R1, distScaled;
MOV_SAT R2, distScaled;
ADD  R1, R1, R2;
MUL  RangeIsInvalid, R1, 0.5;
MUL_SAT R1, -dist, DepthTestSoftness;
LRP  R1, RangeIsInvalid, DefVal, R1;
ADD  ssao, ssao, R1;
###=================================================
DP4  ssao.x, ssao, inv8;
ADD  ssao.x, ssao, -Bias;
MOV  oColor, depthEnc.xxyy;
LRP_SAT oColor.x, Amount, ssao.x, 0.9;
END

[AluminumHaste]

Sikkpins SSAO used a vile hack to get the depth image aside from that it works quite ok but breaks when colliding with some shader effects like the heathaze and skybox shaders.

 

The effect caused outlines to be drawn on alpha materials so whenever there was one of said effects you could see the outlined box.

It looked quite ghastly whenever that happened, but in the places where it worked it looked bloody amazing

 

Yeah we had that with SSAO now, the skybox had it's edges AOed, looked weird.

EDIT: Ohhh, this is so exciting!!!

[Springheel]

Along with all the decals...made it unusable for us.

[demagogue]

Along with the hefty performance hit it takes, but it looks like with this method we can fix at least that much. Anyway SSAO is best for daytime or interior scenes. Doesn't have the same impact at night because it's already dark and SSAO darkens thing further. Looks good when the scene is right for it anyway.

 

IIRC there were 2 other things that must have used the hack because they caused a big performance hit, the bloom and some other feature I can't even remember because we couldn't tell what it was doing just looking at it.

[SteveL]

One question: The zNear and zFar values are hard-coded. How does that work in praxis, are they later to be computed dynamically somehow? Or are these part of the engine, e.g. each engine needs different values?

 

I don't expect they're constant. They'll be computed by the game each frame from the max (and possibly min) distances of visible vertices. I've not looked for that bit of code yet. Perhaps there'll be a way to recover them from the information in openGL's state variables, or if not they could be passed in by the renderer as shaderparms (->glColor) or local environmental variables. As usual, I'll look for an existing method before adding anything new.

 

EDIT: Ohhh, this is so exciting!!!

I agree. I am suffering severely from lack of sleep today

[New Horizon]

So can the work here potentially be used to reduce the high performance impact of soft shadows too?

[AluminumHaste]

the bloom and some other feature I can't even remember because we couldn't tell what it was doing just looking at it.

 

SSIL was the other one. Screen Space Indirect Lighting. That sounds a lot like Radiosity.

 

Yes it does:

 

[AluminumHaste]

So can the work here potentially be used to reduce the high performance impact of soft shadows too?

 

It should reduce the impact of SSAO due to not having to render the scene twice for each display frame. Didn't softshadows do 2-3 renders to jitter the shadow volume edges or something like that?

[New Horizon]

It should reduce the impact of SSAO due to not having to render the scene twice for each display frame. Didn't softshadows do 2-3 renders to jitter the shadow volume edges or something like that?

 

Yeah, I think it had to render the scene a few times to attain a mask to punch out the shadows, blur it, and then apply it back over the original image. If this could reduce that, wow...what an upgrade.

[Tels]

I don't expect they're constant. They'll be computed by the game each frame from the max (and possibly min) distances of visible vertices. I've not looked for that bit of code yet. Perhaps there'll be a way to recover them from the information in openGL's state variables, or if not they could be passed in by the renderer as shaderparms (->glColor) or local environmental variables. As usual, I'll look for an existing method before adding anything new.

 

It was just a question Incidentily, 10 seconds after asking I was about to close the PDF and found that the depth values are available to the shader as option

[motorsep]

SSIL was the other one. Screen Space Indirect Lighting. That sounds a lot like Radiosity.

 

Yes it does:

 

http://www.youtube.com/watch?v=4O49N2-b3CA

 

You need G-Buffer for that. Can't be done with depth image afaik.

[nbohr1more]

Sikkmod has SSIL but whether it works as expected is another story. I think he always referred to it as "just fucking around".

I recall the first version made the scenes render slower than with his soft-shadows effect enabled.

[AluminumHaste]

Yeah I've seen screenshot of people with all 3 turned on, running at 3 fps lol

 

Here is an example of how it looks:

Without:

 

 

With:

 

 

Okay he's doing something weird with image shack, article here: http://renegaderocks...last-of-us.html

 

Here's a screenshot from the website:

 

[demagogue]

Sikkmod has it already, but I imagine it could use another hack because it slows things down quite a bit. It's probably not too visible in TDM again because it's not a bright game, nor all that colorful for that matter. But it might not be functioning for some reason too.

 

But if we can get it with everything else IMO it'd be welcome. We could use radiosity. It makes a space seem realer as much as any effect can. Maybe if we turned it up (or functioning if it's broken) we could see a difference.

[nbohr1more]

The funny thing is that JC Denton's enhanced interaction changes include Maha-X's "brilliant highlights" mod which has some bounced lighting approximation.

I wonder if the two effects would clash? Probably not since real light bounces far more than twice anyway...

[New Horizon]

I'm sure one or the other could be disabled if there were any clashes.