Jump to content
The Dark Mod Forums

Messin with shaders


totallytubular

Recommended Posts

v 0.22

  • BRDF & PBR implemented... in faux fashion

Snagged code from...

...and chucked it in a file called "interaction.pbr.glsl" that gets imported in ambient.fs & common.fs. Can switch it on off by commenting / uncommenting the "FAKE_PBR" define.

Can't tell much of a difference (imo), because using faux Specular & Roughness maps. But, the code is there now. Bit of a hit to FPS, but I'm running TDM on an integrated Intel HD 4600 (my GTX760 crapped out). I'm still getting around 20-30 FPS, though, using stencil shadows & med SSAO.

glprogs.stages.interaction.022.zip

  • Like 4
Link to comment
Share on other sites

1 hour ago, totallytubular said:

You mean this...?

CQ6tD40.jpg

.. where it looks like a rectangle shape that's not taking the shaders into account?

I have no clue what's causing that. I see it randomly. On some levels it's like a massive volumetric fog effect is floating around causing cut-off seams to show up. Other times it's just windows. In Training Mission, some windows are fine, but others have this issue. It feels like some level assets are using old shaders or other shaders than what I'm messing with. I don't know.

Yes.

Ambient lights have two modes:

Omni ( Cube )

Projected ( Pyramid )

You need to handle this toggle.

Please visit TDM's IndieDB site and help promote the mod:

 

http://www.indiedb.com/mods/the-dark-mod

 

(Yeah, shameless promotion... but traffic is traffic folks...)

Link to comment
Share on other sites

1 hour ago, nbohr1more said:

Yes.

Ambient lights have two modes:

Omni ( Cube )

Projected ( Pyramid )

You need to handle this toggle.

Is that all in the "interaction.ambient.fs" shader file?

main() branches with a "if ( u_cubic == 1 )", and I set a "light.r *= 10.0;" on that first branch to see if anything lit up, so I'd know where that branch was being used.

Nothing lit up red.

Made me assume the cubic branch wasn't used. Or at least not used in Training Mission.

Everything I've done in ambient.fs has been in the 2nd part of the branch, and outside the logic branch with fresnels wrap-up and such.

Everything made me assume those windows/lights were tied to some other legacy shader that I couldn't find, or were still being handled by the game engine's built-in graphics or something.

I can mess around with it more. But, I'd like to confirm first that that branch is indeed running those windows, so I'm not wasting time fixing something that isn't the cause. I don't know how to confirm what shader runs those windows.

Link to comment
Share on other sites

light_20

 

"classname" "light"
"name" "light_20"
"_color" "0.09 0.13 0.20"
"light_right" "-85.1239 6.8898e-006 -95.6761"
"light_target" "357.172 -9.12726e-006 -312.14"
"light_up" "-1.05096 -96 -0.946324"
"nodiffuse" "1"
"noshadows" "1"
"nospecular" "1"
"origin" "256 -1904 481"
"parallel" "0"
"texture" "lights/tdm_roundtop_window"

 

light_up  and light_right are projected args

 

(from the map file)

Please visit TDM's IndieDB site and help promote the mod:

 

http://www.indiedb.com/mods/the-dark-mod

 

(Yeah, shameless promotion... but traffic is traffic folks...)

Link to comment
Share on other sites

9 hours ago, nbohr1more said:

"nodiffuse" "1"

That one won't even work, will it? (And possibly "nospecular" didn't either when the light was added, if it predates TDM 2.05.) I wonder what the mapper's intention was in trying to disable diffuse and specular.

Some things I'm repeatedly thinking about...

 

- louder scream when you're dying

Link to comment
Share on other sites

Yeah, this light was setup long ago but other more modern projected (pyramid) lights also are broken with these shaders such as the projected lights in "Caduceus of St Alban"

Please visit TDM's IndieDB site and help promote the mod:

 

http://www.indiedb.com/mods/the-dark-mod

 

(Yeah, shameless promotion... but traffic is traffic folks...)

Link to comment
Share on other sites

3 hours ago, VanishedOne said:

That one won't even work, will it? (And possibly "nospecular" didn't either when the light was added, if it predates TDM 2.05.) I wonder what the mapper's intention was in trying to disable diffuse and specular.

If it's just a light source parked next to a window to make it look like the window is giving off light, then it makes sense. It shouldn't have diffuse color or specular. Heck, it shouldn't have normals or anything. It should just be a light source.

There's windows that have "shine" coming out of them, and those will have diffuse color for the "rays", but they shouldn't do normals or speculars. Should just be a diffuse color piped through with a reduced alpha transparency. They're just simple sfx that don't need as much processing as, say, a texture on a wall or such.

I don't know exactly how the game engine works or how it processes these things. I just know what I've seen in the shaders for TDM, and how I've seen things work in other games.

So, I'm kind of at a loss for figuring out what's causing the pyramid lights to bork up and how to fix it.

Link to comment
Share on other sites

14 hours ago, nbohr1more said:

light_20

 

"classname" "light"
"name" "light_20"
"_color" "0.09 0.13 0.20"
"light_right" "-85.1239 6.8898e-006 -95.6761"
"light_target" "357.172 -9.12726e-006 -312.14"
"light_up" "-1.05096 -96 -0.946324"
"nodiffuse" "1"
"noshadows" "1"
"nospecular" "1"
"origin" "256 -1904 481"
"parallel" "0"
"texture" "lights/tdm_roundtop_window"

 

light_up  and light_right are projected args

 

(from the map file)

 

Conclusion first...

  • "u_cubic" isn't getting set to "1" in game engine, or isn't being piped in properly or something.

Experiment 1:

  • I put original 2.09 shaders back into "interaction" folder.
  • For ambient.fs, set final light col in u_cubic branch to .g*100 and other branch is .b*100.
  • For common.fs, set lightcolor u_cubic branch to .b*100 and other branch to .g*100.

Swapping them should give us night bright overlaps to see how they interact. I get the following...

Vv8iC8T.jpg

Our problem window is not lit up bright blue.

Experiment 2:

  • set common.fs final output to .rgb = vec3(0.0) to nullify it
  • kept ambient *100's the same.

Got the following...

LFCC8xS.jpg

Everything's blue, which means everything is using the != u_cubic branch.

Experiment 3 .. flip it:

  • ambient.fs final output .rgb = vec3(0.0)
  • common.fs *100's returned

NPmMcYm.jpg

Problem window is pitch black when it should be blue.

So, it's using the non-cubic branch, but since that's not pulling the cubic light texture, it's pitch-black I'm guessing.

I did experiments like this when first mucking with the shaders, and got same results. That's why I commented shader code that it seemed like u_cubic branch wasn't used. I never saw color changes for it. Figured it was pre-coded for a future feature enhancement.

But, it looks broken. I don't know where it's broken. I haven't looked at game source code. I just stare at shaders. Someone else more knowledgeable about the game source could look into this?

Link to comment
Share on other sites

Afaik by using the global material keyword "blendLight" in the light material and disabling specular and diffuse you turn the light into a very cheap light, this lights have no real interaction with surfaces this is what iddevnet said about them:

Quote

 blendLight -> Perform simple blending of the projection, instead of interacting with bumps and textures.

 

Edited by HMart
Link to comment
Share on other sites

I should clarify even further.

There are

Omni \ Point lights == Standard lights. 2D projection Image + 1D Falloff Image ( Cube shaped)

Projected Lists == Standard lights that use a pyramid projection shape

Parallel Lights == Same shape as Point Lights by shadows are calculated all parallel to the projection

CubicLights == Uses a Cubemap and projects spherically. This is rarely used and is why you did not see it in your testing

AmbientCubicLights == Same as CubicLights but the shading is meant to use a standard Ambient Cubemap \ IBL image

AmbientLight ( can be either Point or Projected) is a no shadows version of normal lighting that uses a distinct non-directional light shader

https://wiki.thedarkmod.com/index.php?title=Light_Properties

 

I think the GLSL shaders do a NULL W detection to determine whether a light is projected or point.

 

Please visit TDM's IndieDB site and help promote the mod:

 

http://www.indiedb.com/mods/the-dark-mod

 

(Yeah, shameless promotion... but traffic is traffic folks...)

Link to comment
Share on other sites

Experimented more...

Disabled all !u_cubic branch so only u_cubic == 1 branch runs (and leaving everything else outside the branch un-touched).

Got this with the special window in Training Mission...

foeUGtq.jpg

nothing. Window doesn't show up.

Then I disabled all output from both ambient.fs and common.fs (set their final rgb's to 0) to see what shows up.

Got the following...

xuGf9rf.jpg

So, some windows are bypassing the ambient/common lighting shaders to act as a pass through.

It seems like that special window should too?

I think there are some windows that do use ambient/common lighting shaders, because they're doing specular effects (shine off glass), etc.

But, I'm not sure what's going on with those windows having issues. It's like they should be bypassing, but aren't. But, are not sending a flag or textures or something to the shaders to use. (IE: because they could be calling the shaders correctly, but if the texture they're trying to call and use isn't set, it would return an rgb color of 0.0 which would make it seem like it's not working.)

Again.. total noob here.. all I've stared at is shaders. Don't know how game engine works, or what it's piping to shaders.

Link to comment
Share on other sites

I'll take another look tonight.

I actually fixed this issue once in 2.06 by simply adding the falloff image back to the shaders...

but the GLSL currently is very arcane now so it's hard to tell where the "projected light" branch I'm missing is.

@duzenko ?

Please visit TDM's IndieDB site and help promote the mod:

 

http://www.indiedb.com/mods/the-dark-mod

 

(Yeah, shameless promotion... but traffic is traffic folks...)

Link to comment
Share on other sites

3 hours ago, HMart said:

Afaik by using the global material keyword "blendLight" in the light material and disabling specular and diffuse you turn the light into a very cheap light, this lights have no real interaction with surfaces

 

Yes, what that keyword does is let you put blend stages in the light material instead of regular light interaction stages. E.g. blend add in glare lights, blend filter in shadow lights. (And no shadowcasting; id's docs say forceshadows should override that like it does for ambient lights, but in my tests it didn't.)

 

5 hours ago, totallytubular said:

If it's just a light source parked next to a window to make it look like the window is giving off light, then it makes sense. It shouldn't have diffuse color or specular. Heck, it shouldn't have normals or anything. It should just be a light source.

There's windows that have "shine" coming out of them, and those will have diffuse color for the "rays", but they shouldn't do normals or speculars. Should just be a diffuse color piped through with a reduced alpha transparency. They're just simple sfx that don't need as much processing as, say, a texture on a wall or such.

As far as I know the point of nodiffuse/nospecular is to disable things the light does when illuminating surfaces in the world. Nothing to do with SFX.

Some things I'm repeatedly thinking about...

 

- louder scream when you're dying

Link to comment
Share on other sites

v 0.23

  • realized BRDF diffuse & specular outputs are good-to-go as-is, and just needed to add together then mul by lightcolor, so adjusted end routine for BRDF to do so w/o running them through light dots further
  • shuffled code around in ambient/common lighting to skip calc'ing stuff not needed for BRDF
  • switched to dot'ing fake AO/Spec maps by a luminosity rgb instead of 0.33 to try to make them a more natural grayscale
  • figured out (I think) how to apply fake AO to ambient BRDF to impact both diffuse and specular

BRDF is uncharted territory for me, so I'm still learning as I go. The github repo I pulled the code from had the BRDF / PBR parts, but didn't explain how to apply AO and such. I stumbled across this site...

https://google.github.io/filament/Filament.md.html

... that talks in-depth about stuff, and has a lot of GLSL code snippets. That's where I read-up on the AO.

I hard-coded these shaders to use the most "complex" diffuse and specular BRDF functions from the github repo I first used. But, since there's lighter versions of the functions, BRDF could end up with "levels", like SSAO, where end-user could choose how complex of calculations they want. That's something to think about later, though.

I'm actually really impressed by the BRDF functions.. not in that they make everything look good (I'm not sure I notice that big of a difference). But, more in that it encapsulates pretty much everything you want to do with lighting. It processes the specular via fresnel, it applies the spec dot to specular, it runs diffuse through a function that applies light dot.. that's at it's most basic. It goes into much more detail in what it does, but it handles a lot of the stuff the amb/common light shaders were doing. Just have to calculate the normals, directions, light color, etc, then shove it through the BRDF in a line of code, and out pops the end-result diffuse & specular that are ready to just add together, mul by light color.

glprogs.stages.interaction.023.zip

  • Like 2
Link to comment
Share on other sites

One thing I didn't think about until just now... licensed usage of code I plugged in.

I looked at the urho3D github for a license for the BRDF / PBR ... they have a very open-ended MIT license...

https://github.com/urho3d/Urho3D/blob/master/LICENSE

just asks that the copyright be included with the code. I can paste that into the "interactions.pbr.glsl" code file no problem.

 

The code snippet I copied from the Filament github for specular AO... I went up the tree on their project, and they have an Apache license.

https://github.com/google/filament/blob/main/LICENSE

Section 4 talks about redistribution of code with several steps to provide notice and attribution. Not quite sure how to do that. Not sure if that's something to think about now or later .. or at all?

Since TDM is a big project, just trying to dot the i's and cross the t's to make sure things are done in a semi-official manner to avoid issues.

  • Like 1
Link to comment
Share on other sites

8 hours ago, peter_spy said:

Why using some Urine3d when you can have an industry standard that is Disney BRDF, which is also open source, IIRC?

https://github.com/wdas/brdf

I'd never heard of BRDF until I got on these forums, so didn't know where to begin.

I came across the OpenGL theory page that talked about BRDF, and, while it gave code snippets, they didn't put it all together the way urho3D did. Urho3D's version was the first complete version I stumbled across that was simple and straight-forward when random googling. Their implementation of BRDF was easy to understand and plug-n-play in-a-day. Right now I'm just doing some quick-n-dirty messing around to see if BRDF teamed with the fake AO/spec maps does anything magical.

I'm looking at Disney's a bit and it's ... pretty complex. At least for me. They've got GLSL mixed in with C code to do a whole application to explore things. Would take time to dig in and figure out what parts to use.

If real BRDF / PBR was being implemented in TDM, it'd be good to look at Disney's version. But, for now, I'm just messing around. (I'm killing time farting around on this while I'm looking for a job, so I don't go stir crazy.)

Disney's license looks pretty open, though.

TDM has a LICENSE.txt file. Could just chuck the 3rd-party licenses into that as sections, then in the code make comments like "this code pulled from (web-site), and has license in TDM's LICENSE.txt file, section XYZ".

Didn't mean to derail the topic talking about licensing. Just something I suddenly thought about after pasting in and using tons of code I found and going "crap, this is pointless if there's going to be a license issue."

  • Like 1
Link to comment
Share on other sites

(tl;dr version)

FAKE_PBR is actually not working. The shaders don't seem to register the #pragma tdm_define flags I set, so it's been running the non-FAKE_PBR branch this whole time. (and non-BLINN_SPECULAR, which means it's doing phong speculars).

Anyone know where TDM dumps GLSL compile output log or something, so I can debug what's blowing up?

 

 

(long version)

Was modifying code to make it easier to switch between different fresnel, visibility, etc func's. Added red-highlighting to BRDF spec output in a func to make sure branching was working. Nothing was highlighting. Went up the chain to the BRDF function to highlight outputs .r * 100 ... nothing. Went into amb/common to highlight the light output .r from BRDF branch..nothing.

Finally realized that the code's been running the !FAKE_PBR branch the whole time.

I replaced the ...

  • #pragma tdm_define "FAKE_PBR"

... with...

  • #define FAKE_PBR

.. to see if I could force the define that way. Nope.

I switched the code to use bool constants for PBR & BLINN... nope, everything in game world decides to be dark, which means ambient & common fs shaders blowing up during compile.

I've been commenting out large chunks of code to cut out PBR to just get back to something that will compile and work, so I could go from there.. but something's so borked-up I can't figure out what's borked up.

I'd rather not waste time guessing what's borked up. Would rather see GLSL compiler output, so I have some clue what to look for. Not sure how to do that.

Link to comment
Share on other sites

Open the console ( CTRL + ALT + Tilde ) then invoke r_debugGLSL 1 and after that invoke reloadGLSLprograms

If you want it in a text file then invoke condump <name_of_file> and it will write to your darkmod directory.

You can leave the GLSL file open and have TDM in a window then save your file and invoke reloadGLSLprograms and watch the results in real-time.

  • Like 1

Please visit TDM's IndieDB site and help promote the mod:

 

http://www.indiedb.com/mods/the-dark-mod

 

(Yeah, shameless promotion... but traffic is traffic folks...)

Link to comment
Share on other sites

5 hours ago, nbohr1more said:

Open the console ( CTRL + ALT + Tilde ) then invoke r_debugGLSL 1 and after that invoke reloadGLSLprograms

If you want it in a text file then invoke condump <name_of_file> and it will write to your darkmod directory.

You can leave the GLSL file open and have TDM in a window then save your file and invoke reloadGLSLprograms and watch the results in real-time.

Holy s*** ... that is like umpteen times better than what I was doing!

I'd modify shader code, start game, load Training Mission (since seeing the same level over and over makes it easy to spot differences), check results, close game... wash-rinse-repeat. It would take a minute or so for Training Mission to load, so it was painful double-checking things, especially when having to chase down something I screwed up.

Without a debug log, I'd have to scrub the code for mistakes. I got pretty good at that after working on the FUEL shaders, b/c they didn't have a debug output either. But, sometimes things get so borked up you just can't tell what's hosed w/o a debug log.

Like with PBR.. I was pulling my hair out on that.

I figured out the issue with it before reading this, though. (wish I would have waited). Basically did massive "#ifdef SKIPCODE" to bypass large chunks of code until I could narrow down the issue. There was an undeclared var in one of the PBR sub-routines due to a typo I made.

smh

So, got PBR working, and it looks ugly. It's making things really dark. So, now I'm testing / tweaking it. (Mainly tweaking the output of it.. once diffuse & specular are modified by BRDF, have to figure out how to work lightcolor in and combine it all to look decent.)

I'll hopefully upload something later today.

Can't believe I went a whole week assuming the PBR code was working when it wasn't.

I kept playing levels and looking and going "man, is there a difference? I don't know. Is it placebo effect? I don't know."

smh

Rookie mistake.

Link to comment
Share on other sites

v 0.26

  • BRDF works now

Switched to using regular #defines for it. Had to mess with it a lot.

urdo3D code makes it seem like you just...

  • (diffuse + specular ) * lightcolor

... the BRDF result, and you're on your way. But, doing that, I was getting ugly results. Normals were flat. Specular was non-existent. The diffuse terms BRDF uses don't take light dot into account (except for the burley function that uses it more for a tweak).

After fiddling around, decided to just let diffuse & specular come out of the BRDF function, and finish going through the lighting gauntlet. So, BRDF-modified specular gets impacted by a spec dot, BRDF-modified diffuse gets impacted by a light dot.. both get light color merged into them.. etc.

It was the only way I could get decent-looking results. Feels like I'm doing something wrong doing that, though.

Using inverted grayscale specular to generate fake roughness was nullifying speculars, so I switched over to using dot(diffuse) for roughness. It gets better results in that the specular shows up now.. but it's a bit shiny in places.

The BRDF is not giving "omg, wow!" results... mostly b/c using normals & diffuse to generate fake maps simply reinforces the those textures. (EG: if you keep using diffuse for every fake map, eventually speculars look super wet.. b/c all you're doing is just reinforcing that texture pattern over and over).

Anyways... here's the Builder sneak area in Training Mission using the non-PBR shader branch...

uuMGxkx.jpg

And here's PBR shader branch...

I8sCsNC.jpg

The burley diffusion term punches up diffuse coloring. The diffuse map generating a faux roughness map helps it impact speculars... the grating on the floor in lower-right has specular shine more in tune with the grating color. There's shine on the wall in the distance from the torch now.. but that can be accomplished by just ramping up specular. But, the specular terms that PBR uses decides to boost and dim specular in various spots, so the scene ends up with shines in areas that didn't have them before. (Not sure if that's a good or bad thing... EG: I noticed a lot of specular seems to hang around lower wall now with PBR on. Not sure why.)

 

Here's the melee / archery outdoor area... no PBR...

QYwqTRi.jpg

Notice the clock tower windows...

Now notice them when PBR's on...

PGvJ7oM.jpg

PBR makes the shine on the clock tower look more natural, but it negated the shine on the windows. Not sure if it's the diffuse pbr term or one of the specular terms that's doing it.

~~~~~~~~~~~~~~

The problem with the BRDF / PBR stuff is there's several terms of each type to experiment with, and they can give different results.

EG: using the Lambertian diffuse terms, you get dark results. Using the Burley diffuse term, you get results more in sync with the non-PBR colors.

So, the catch-22 is that you can end up having to code up a bunch of "if using this term, use this crutch to make up for XYZ that it does" (eg: some specular terms cause LOTS of shine.. others dull things down).

I experimented with combinations, and came up with a "preset" that seems to do ok. That was only after I switched to using diffuse as roughness. When I had inverted specular as roughness, I was pulling my hair out for an hour trying to figure out why specular wasn't showing up at all.

This was an interesting experiment, but not sure fake PBR / BRDF really adds enough value to be worth it. It's a huge PITA to have to tweak everything.

glprogs.stages.interaction.026.zip

  • Like 1
Link to comment
Share on other sites

There's no point in introducing PBR if you're not doing it on correct data. Creating a simple scene in Unreal of Unity, and then using the same images and materials in TDM scene would give more meaningful results to compare.

Edited by peter_spy
Link to comment
Share on other sites

Some pretty strange results in the latest build:

1) If you enable Bloom your screen will be filled with black squares that have rounded edges, similar to this thread:

but little boxes everywhere

2) Similarly, r_tonemap produces tiny black artifacts near light sources

3) Some speculars near yellow light sources are blinding white for some reason

Please visit TDM's IndieDB site and help promote the mod:

 

http://www.indiedb.com/mods/the-dark-mod

 

(Yeah, shameless promotion... but traffic is traffic folks...)

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recent Status Updates

    • Petike the Taffer

      I've finally managed to log in to The Dark Mod Wiki. I'm back in the saddle and before the holidays start in full, I'll be adding a few new FM articles and doing other updates. Written in Stone is already done.
      · 0 replies
    • nbohr1more

      TDM 15th Anniversary Contest is now active! Please declare your participation: https://forums.thedarkmod.com/index.php?/topic/22413-the-dark-mod-15th-anniversary-contest-entry-thread/
       
      · 0 replies
    • JackFarmer

      @TheUnbeholden
      You cannot receive PMs. Could you please be so kind and check your mailbox if it is full (or maybe you switched off the function)?
      · 1 reply
    • OrbWeaver

      I like the new frob highlight but it would nice if it was less "flickery" while moving over objects (especially barred metal doors).
      · 4 replies
    • nbohr1more

      Please vote in the 15th Anniversary Contest Theme Poll
       
      · 0 replies
×
×
  • Create New...