dhewm3 with some extras

[revelator]

so after not touching game related code for some years i finally decided to do something that might benefit others.

i have a build of dhewm3 with my hybrid GLSL ARB2 renderer and your AVX/AVX2 Simd functions.

The GLSL renderer only handles interactions, shadows and user shaders are still ARB2 so it is 90% compatible with mods like sikkmod (the 10% only relates to sikkmods interaction shaders like pom so if you want to use that turn off the GLSL renderer).

dhewm3 also uses older gamma correction and soft particle shaders from TDM which also interfere with sikkmod but can also be turned off.

the SDL cpu capability detection is largely broken on windows so i rolled my own otherwise it allways defaults to the generic Simd path (might be a problem in how the detection is used ?).

if anyone wants to try it out before i commit it let me know.

[lowenz]

Oh!

[revelator]

the hybrid renderer could probably be modified to also handle shadows in GLSL .

im also using an RAII style depthbounds checker for both the shadows and lighting.

the gamma correction works mostly with sikkmod but there is one oddity i noticed. while the world looks fine with it character faces go way bright.

not sure what problems the softparticles have but it was noted by the dev himself that some stuff might look wrong with it.

the VBO code was also replaced by a more efficient version originally written by michael hiney from insideqc. It is not a total rewrite like the version in TDM that uses the BFG version, instead it uses MapBufferRange and some optimizations.

the GLSL hybrid renderer was mainly made to allow older mods to still work and it is quite solid by now having ironed out the last bugs with it years ago.

[revelator]

Spoiler

 

/*
===========================================================================

DEFAULT GLSL *SHADER

===========================================================================
*/
#define GLSL_VERSION_ATTRIBS "#version 130\n"

#define GLSL_INPUT_ATTRIBS                                                                                             \
    "in vec4 attrTexCoords;\n"                                                                                         \
    "in vec3 attrTangents0;\n"                                                                                         \
    "in vec3 attrTangents1;\n"                                                                                         \
    "in vec3 attrNormal;\n"                                                                                            \
    "mat3x3 u_lightMatrix = mat3x3 (attrTangents0, attrTangents1, attrNormal);\n\n"

#define GLSL_UNIFORMS                                                                                                  \
    "uniform vec4 u_light_origin;\n"                                                                                   \
    "uniform vec4 u_view_origin;\n"                                                                                    \
    "uniform vec4 u_color_modulate;\n"                                                                                 \
    "uniform vec4 u_color_add;\n"                                                                                      \
    "uniform mat2x4 u_diffMatrix;\n"                                                                                   \
    "uniform mat2x4 u_bumpMatrix;\n"                                                                                   \
    "uniform mat2x4 u_specMatrix;\n"                                                                                   \
    "uniform mat4x4 u_projMatrix;\n"                                                                                   \
    "uniform mat4x4 u_fallMatrix;\n"                                                                                   \
    "uniform sampler2D bumpImage;\n"                                                                                   \
    "uniform sampler2D lightFalloffImage;\n"                                                                           \
    "uniform sampler2D lightProjectImage;\n"                                                                           \
    "uniform sampler2D diffuseImage;\n"                                                                                \
    "uniform sampler2D specularImage;\n"                                                                               \
    "uniform vec4 u_constant_diffuse;\n"                                                                               \
    "uniform vec4 u_constant_specular;\n\n"

#define GLSL_VARYINGS                                                                                                  \
    "varying vec2 diffCoords;\n"                                                                                       \
    "varying vec2 bumpCoords;\n"                                                                                       \
    "varying vec2 specCoords;\n"                                                                                       \
    "varying vec4 projCoords;\n"                                                                                       \
    "varying vec4 fallCoords;\n"                                                                                       \
    "varying vec3 lightDir;\n"                                                                                         \
    "varying vec3 halfAngle;\n"                                                                                        \
    "varying vec4 Color;\n"

// these are our GLSL interaction shaders
#define interaction_vs                                                                                                 \
    GLSL_VERSION_ATTRIBS                                                                                               \
    GLSL_INPUT_ATTRIBS                                                                                                 \
    GLSL_UNIFORMS                                                                                                      \
    GLSL_VARYINGS                                                                                                      \
    "void main ()\n"                                                                                                   \
    "{\n"                                                                                                              \
    "	// we must use ftransform as Doom 3 needs invariant position\n"                                                \
    "	gl_Position = ftransform ();\n"                                                                                \
    "\n"                                                                                                               \
    "	diffCoords = attrTexCoords * u_diffMatrix;\n"                                                                  \
    "	bumpCoords = attrTexCoords * u_bumpMatrix;\n"                                                                  \
    "	specCoords = attrTexCoords * u_specMatrix;\n"                                                                  \
    "\n"                                                                                                               \
    "	projCoords = gl_Vertex * u_projMatrix;\n"                                                                      \
    "	fallCoords = gl_Vertex * u_fallMatrix;\n"                                                                      \
    "\n"                                                                                                               \
    "	Color = (gl_Color * u_color_modulate) + u_color_add;\n"                                                        \
    "\n"                                                                                                               \
    "	vec3 OffsetViewOrigin = (u_view_origin - gl_Vertex).xyz;\n"                                                    \
    "	vec3 OffsetLightOrigin = (u_light_origin - gl_Vertex).xyz;\n"                                                  \
    "\n"                                                                                                               \
    "	lightDir = OffsetLightOrigin * u_lightMatrix;\n"                                                               \
    "	halfAngle = (normalize (OffsetViewOrigin) + normalize (OffsetLightOrigin)) * u_lightMatrix;\n"                 \
    "}\n\n"

#define interaction_fs                                                                                                 \
    GLSL_VERSION_ATTRIBS                                                                                               \
    GLSL_UNIFORMS                                                                                                      \
    GLSL_VARYINGS                                                                                                      \
    "void main ()\n"                                                                                                   \
    "{\n"                                                                                                              \
    "	vec3 normalMap = texture2D (bumpImage, bumpCoords).agb * 2.0 - 1.0;\n"                                         \
    "	vec4 lightMap = texture2DProj (lightProjectImage, projCoords);\n"                                              \
    "\n"                                                                                                               \
    "	lightMap *= dot (normalize (lightDir), normalMap);\n"                                                          \
    "	lightMap *= texture2DProj (lightFalloffImage, fallCoords);\n"                                                  \
    "	lightMap *= Color;\n"                                                                                          \
    "\n"                                                                                                               \
    "	vec4 diffuseMap = texture2D (diffuseImage, diffCoords) * u_constant_diffuse;\n"                                \
    "	float specularComponent = clamp ((dot (normalize (halfAngle), normalMap) - 0.75) * 4.0, 0.0, 1.0);\n"          \
    "\n"                                                                                                               \
    "	vec4 specularResult = u_constant_specular * (specularComponent * specularComponent);\n"                        \
    "	vec4 specularMap = texture2D (specularImage, specCoords) * 2.0;\n"                                             \
    "\n"                                                                                                               \
    "	gl_FragColor = (diffuseMap + (specularResult * specularMap)) * lightMap;\n"                                    \
    "}\n\n"

/* 32 bit hexadecimal 0, BFG had this set to a negative value which is illegal on unsigned */
static const GLuint INVALID_PROGRAM = 0x00000000;

static GLuint u_light_origin = INVALID_PROGRAM;
static GLuint u_view_origin = INVALID_PROGRAM;

static GLuint u_color_modulate = INVALID_PROGRAM;
static GLuint u_color_add = INVALID_PROGRAM;

static GLuint u_constant_diffuse = INVALID_PROGRAM;
static GLuint u_constant_specular = INVALID_PROGRAM;

static GLuint u_diffMatrix = INVALID_PROGRAM;
static GLuint u_bumpMatrix = INVALID_PROGRAM;
static GLuint u_specMatrix = INVALID_PROGRAM;

static GLuint u_projMatrix = INVALID_PROGRAM;
static GLuint u_fallMatrix = INVALID_PROGRAM;

static GLuint rb_glsl_interaction_program = INVALID_PROGRAM;

The version i modified for dhewm3 in case someone else wants to toy with it

the above goes into tr_local.h

the baked in GLSL shader code can be overridden with external shaders to.

commented a few things like the remnant of the render selection path which is a bit redundant since dhewm3 only supports ARB2 (well GLSL as well but the code does not know that )

[revelator]

Spoiler

/*
============================================================================

DEPTH BOUNDS TESTING

============================================================================
*/

DepthBoundsTest::DepthBoundsTest( const idScreenRect &scissorRect ) {
	if ( !glConfig.depthBoundsTestAvailable || !r_useDepthBoundsTest.GetBool() ) {
		return;
	}
	assert( scissorRect.zmin <= scissorRect.zmax );
	qglEnable( GL_DEPTH_BOUNDS_TEST_EXT );
	qglDepthBoundsEXT( scissorRect.zmin, scissorRect.zmax );
}

DepthBoundsTest::~DepthBoundsTest() {
	if ( !glConfig.depthBoundsTestAvailable || !r_useDepthBoundsTest.GetBool() ) {
		return;
	}
	qglDisable( GL_DEPTH_BOUNDS_TEST_EXT );
}

 

this one is the RAII style depthbounds test, it goes into tr_backend.cpp

Spoiler

// RAII-style wrapper for glDepthBoundsEXT
class DepthBoundsTest {
public:
	DepthBoundsTest( const idScreenRect &scissorRect );
	~DepthBoundsTest();
};

 

and this goes into tr_local.h

notice it is also used in draw_common.cpp in the function above.

so remember to remove the old depthboundstest from RB_StencilShadowPass

 

[nbohr1more]

Cool stuff!

If you wouldn't mind, maybe submit a PR to the Dhewm3 git repo for the AVX \ SIMD things?

( Not sure Daniel will go for the GLSL changes )

[revelator]

aye though the GLSL renderer is wholly unobtrusive and can be disabled completely if anything barks   leaving just the ARB2 renderer.

Ill see what he says to it .

[revelator]

https://sourceforge.net/projects/cbadvanced/files/Game Projects/dhewm3.7z/download

try it out tested it with sikkmod classic doom sikkmodd3xp lost mission so far it runs quite fine on my machine but lets be sure before i start pestering Daniel hehe.

omg just ran the built in benchmark and i get 3199 fps in ultra with the HD texture pack

[revelator]

https://github.com/dhewm/dhewm3/issues/630

posted a build with the changes lets see what he thinks.

[The Black Arrow]

Hell yeah, I always wanted to play Doom 3 with the many improvements that TDM made, for example...Faster loading times, soft shadows using Maps instead of Stencils, better FPS interpolation (no longer needing a 60 FPS limit that way), true/better OpenAL support, better color precision...Even the Bloom!

I hope it becomes official in dhewm3!

[revelator]

well he sounds interrested so crossing fingers .

shadowmap code could be interresting to add but that one requires even more GLSL code and most likely the addition of framebuffers so im not sure i could sell that one to him.

[revelator]

going to try porting the grimm mod for dhewm3 next since many seems to want it and i have his permission to use it.

[revelator]

how it looks with highres textures, strangely gamma does not seem to affect the screenshot with dhewm3 ?.

[revelator]

Had some buggers with SDL's cpu detection code changing the denormals are zero detection code to TDM's fixed it strangely huh ?.

i also added TDM's detection code for SSE3 AVX and AVX2 in case someone wants to build it using the old SDL1.2 since it only has detection code upto SSE2.

The old 3DNow code was still in there as well but is unsupported in the upcomming SDL3 (also disabled by Daniel).

ALTIVEC... groan but SDL does support it.

sikkmod and sikkmodd3xp was ported to dhewm3 so that's a plus sikkmodd3xp has a bug though which makes monsters nearly unkillable if firering a weapon head on (only splash damage seems to work). It also uses a somewhat older codebase than sikkmod for the base game (i actually updated the code to the later sikkmod version some years back but not for dhewm3, gonna have to try and get it ported).

[revelator]

grimm_2.zip

the game dll for grimms quest for the gatherers key ported to dhewm3.

enjoy

[revelator]

dhewm has a built in imgui gui for setting stuff that the normal menu does not have (well most things can be set in it actually) hit F10 while ingame to open it.

might add the GLSL backend renderer to it as well for easy disabling.

with grimms quest you should disable the softparticles cause they interfere with the sikkmod particle code.

also disable shader gamma otherwise the game looks way to dark.

GLSL works fine but if you want to use sikkpins pom interaction shader you need to disable it.

code was prettyy nasty to port so i hope to god noone asks me to do that again

[datiswous]

@revelatorcould you put the code postings inside a spoiler, so the page scrolls a bit faster?

For example:

Spoiler

/*
===========================================================================

DEFAULT GLSL *SHADER

===========================================================================
*/
#define GLSL_VERSION_ATTRIBS "#version 130\n"

#define GLSL_INPUT_ATTRIBS                                                                                             \
    "in vec4 attrTexCoords;\n"                                                                                         \
    "in vec3 attrTangents0;\n"                                                                                         \
    "in vec3 attrTangents1;\n"                                                                                         \
    "in vec3 attrNormal;\n"                                                                                            \
    "mat3x3 u_lightMatrix = mat3x3 (attrTangents0, attrTangents1, attrNormal);\n\n"

#define GLSL_UNIFORMS                                                                                                  \
    "uniform vec4 u_light_origin;\n"                                                                                   \
    "uniform vec4 u_view_origin;\n"                                                                                    \
    "uniform vec4 u_color_modulate;\n"                                                                                 \
    "uniform vec4 u_color_add;\n"                                                                                      \
    "uniform mat2x4 u_diffMatrix;\n"                                                                                   \
    "uniform mat2x4 u_bumpMatrix;\n"                                                                                   \
    "uniform mat2x4 u_specMatrix;\n"                                                                                   \
    "uniform mat4x4 u_projMatrix;\n"                                                                                   \
    "uniform mat4x4 u_fallMatrix;\n"                                                                                   \
    "uniform sampler2D bumpImage;\n"                                                                                   \
    "uniform sampler2D lightFalloffImage;\n"                                                                           \
    "uniform sampler2D lightProjectImage;\n"                                                                           \
    "uniform sampler2D diffuseImage;\n"                                                                                \
    "uniform sampler2D specularImage;\n"                                                                               \
    "uniform vec4 u_constant_diffuse;\n"                                                                               \
    "uniform vec4 u_constant_specular;\n\n"

#define GLSL_VARYINGS                                                                                                  \
    "varying vec2 diffCoords;\n"                                                                                       \
    "varying vec2 bumpCoords;\n"                                                                                       \
    "varying vec2 specCoords;\n"                                                                                       \
    "varying vec4 projCoords;\n"                                                                                       \
    "varying vec4 fallCoords;\n"                                                                                       \
    "varying vec3 lightDir;\n"                                                                                         \
    "varying vec3 halfAngle;\n"                                                                                        \
    "varying vec4 Color;\n"

// these are our GLSL interaction shaders
#define interaction_vs                                                                                                 \
    GLSL_VERSION_ATTRIBS                                                                                               \
    GLSL_INPUT_ATTRIBS                                                                                                 \
    GLSL_UNIFORMS                                                                                                      \
    GLSL_VARYINGS                                                                                                      \
    "void main ()\n"                                                                                                   \
    "{\n"                                                                                                              \
    "	// we must use ftransform as Doom 3 needs invariant position\n"                                                \
    "	gl_Position = ftransform ();\n"                                                                                \
    "\n"                                                                                                               \
    "	diffCoords = attrTexCoords * u_diffMatrix;\n"                                                                  \
    "	bumpCoords = attrTexCoords * u_bumpMatrix;\n"                                                                  \
    "	specCoords = attrTexCoords * u_specMatrix;\n"                                                                  \
    "\n"                                                                                                               \
    "	projCoords = gl_Vertex * u_projMatrix;\n"                                                                      \
    "	fallCoords = gl_Vertex * u_fallMatrix;\n"                                                                      \
    "\n"                                                                                                               \
    "	Color = (gl_Color * u_color_modulate) + u_color_add;\n"                                                        \
    "\n"                                                                                                               \
    "	vec3 OffsetViewOrigin = (u_view_origin - gl_Vertex).xyz;\n"                                                    \
    "	vec3 OffsetLightOrigin = (u_light_origin - gl_Vertex).xyz;\n"                                                  \
    "\n"                                                                                                               \
    "	lightDir = OffsetLightOrigin * u_lightMatrix;\n"                                                               \
    "	halfAngle = (normalize (OffsetViewOrigin) + normalize (OffsetLightOrigin)) * u_lightMatrix;\n"                 \
    "}\n\n"

#define interaction_fs                                                                                                 \
    GLSL_VERSION_ATTRIBS                                                                                               \
    GLSL_UNIFORMS                                                                                                      \
    GLSL_VARYINGS                                                                                                      \
    "void main ()\n"                                                                                                   \
    "{\n"                                                                                                              \
    "	vec3 normalMap = texture2D (bumpImage, bumpCoords).agb * 2.0 - 1.0;\n"                                         \
    "	vec4 lightMap = texture2DProj (lightProjectImage, projCoords);\n"                                              \
    "\n"                                                                                                               \
    "	lightMap *= dot (normalize (lightDir), normalMap);\n"                                                          \
    "	lightMap *= texture2DProj (lightFalloffImage, fallCoords);\n"                                                  \
    "	lightMap *= Color;\n"                                                                                          \
    "\n"                                                                                                               \
    "	vec4 diffuseMap = texture2D (diffuseImage, diffCoords) * u_constant_diffuse;\n"                                \
    "	float specularComponent = clamp ((dot (normalize (halfAngle), normalMap) - 0.75) * 4.0, 0.0, 1.0);\n"          \
    "\n"                                                                                                               \
    "	vec4 specularResult = u_constant_specular * (specularComponent * specularComponent);\n"                        \
    "	vec4 specularMap = texture2D (specularImage, specCoords) * 2.0;\n"                                             \
    "\n"                                                                                                               \
    "	gl_FragColor = (diffuseMap + (specularResult * specularMap)) * lightMap;\n"                                    \
    "}\n\n"

/* 32 bit hexadecimal 0, BFG had this set to a negative value which is illegal on unsigned */
static const GLuint INVALID_PROGRAM = 0x00000000;

static GLuint u_light_origin = INVALID_PROGRAM;
static GLuint u_view_origin = INVALID_PROGRAM;

static GLuint u_color_modulate = INVALID_PROGRAM;
static GLuint u_color_add = INVALID_PROGRAM;

static GLuint u_constant_diffuse = INVALID_PROGRAM;
static GLuint u_constant_specular = INVALID_PROGRAM;

static GLuint u_diffMatrix = INVALID_PROGRAM;
static GLuint u_bumpMatrix = INVALID_PROGRAM;
static GLuint u_specMatrix = INVALID_PROGRAM;

static GLuint u_projMatrix = INVALID_PROGRAM;
static GLuint u_fallMatrix = INVALID_PROGRAM;

static GLuint rb_glsl_interaction_program = INVALID_PROGRAM;

 

 

[revelator]

sure just forgot

Spoiler

EDIT: Done

[revelator]

Added the GLSL backend renderer to imgui for easy disabling.

added several mod dll's as well as wrappers to start the mods directly (classic doom, lost mission, dentonmod, sikkmod, sikkmod-d3xp, grimms quest for the gatherers key).

https://sourceforge.net/projects/cbadvanced/files/Game Projects/dhewm3-1.53-beta.7z/download

if you want a really good looking doom3 without sikkmod or denton try this ->

https://www.moddb.com/mods/d3hdp-doom-3-essential-hd-pack/downloads/d3hdp-doom-3-essential-hd-pack-v20

it is incompatible with sikkmod so remove the zzz paks if you prefer that.

includes some enhanced particle effects, hd weapons, and mod support for lost mission and the expansion pack. Textures are pretty high quality to.

[revelator]

Ah ok the shader based gamma does not currently work with the GLSL backend (bit strange since the code for it is located in the ARB2 materials section and should still render in ARB2)

well atleast untill i found out it also uses some code in RB_ARB2_DrawInteraction and i currently dont have any GLSL counterpart for RB_GLSL_DrawInteraction.

something to work toward methinks.

[revelator]

grimm was properly ported by daniel while mine worked in x86 it kinda broke in x64, should be up now on github.

as for my dhewm port i need to setup a github account again and diff the changes made but then daniel sees no problems with admitting it.

changes since last time disabled the GLSL backend renderer, can be enabled in the imgui gui.

if GLSL backend is enabled it also disables daniels shader based hardware gamma and softparticles as the GLSL backend lacks shaders for these (should also be disabled if using sikkmod as they break SSAO and DOF).

[revelator]

Well looks quite purty allready screenshots in dhewm are sadly rather dark (ignores gamma / brightness) but grimm looks like a million bucks with it now. 

Allmost to bad that there are not many total conversion games on idtech 4, it actually makes hack and slash games look pretty.

honestly i would love for some cthulhu game in the spririt of dark corners on it, damn that would be scary as hell .

[revelator]

saving current pull as a draft untill i can get the code building on all archs (currently fails on linux / mac).

when things are ironed out ill make a new pull and start adding in only the changes nessesary (daniel dislikes whitespace changes for me it is more a gripe of readability but to each his own).

some changes like the hybrid renderer might have to many changes to totally avoid it though.

also need some way to get the shader based gamma / brightness code ported to the GLSL renderer, i do have a rough idea about what is needed but i generally suck at shaders so lets see how that goes... help! is appreciated.

normally the hybrid renderer can handle both GLSL and ARB at the same time as long as the ARB shader in question is not an interaction shader, sadly as it is setup it is so bummer.

probably also going to experiment with getting the shadow renderer ported to GLSL, this shouldnt be to hard but lets see.

[revelator]

well everything except mac builds build now...

for mac the __cpuid is quite different as it seems to reserve the ebx register so we have to hack around that argh...

slowly got rid of whitespace changes in the code also.

ill probably have to add a few new forks as daniel prefers to have the GLSL hybrid renderer in a seperate branch.

i guess i should have known it would not be that easy since i doubt darkmod was ever ported to mac

[nbohr1more]

The last Mac build we had working was 2.05