Tweaks to Frob highlight

[greebo]

@Tels: Well, I got the impression that a parser would be too much work for you, but I'm happy to wait if you want to do it.

 

Ah, ok, I see you've edited your previous post, so I must have missed that additional part. Well, let me know how this turns out.

 

(And nobody is implementing anything "overnight", you also know that unless somebody starts doing something, nothing will happen. That's why I try to push this topic. Don't feel rushed.)

[OrbWeaver]

I don't know perl and, after looking into the file, I lost the will to learn it.

 

Perl tends to have that effect on people. I made some half-hearted attempts to learn it once, before giving up and deciding that it would be easier just to stick with Python.

[Tels]

Questions:

 

* What about materials that do currently not have a frob-highlight stage, should they still get one?

* no specularmap => _black, but what if it has no bumpmap? What if it has a bumpmap that is calculated by a mapexpression? Same for calculated diffusemaps.

[Baal]

* no specularmap => _black, but what if it has no bumpmap?

Use the _flat keyword in this case.

What if it has a bumpmap that is calculated by a mapexpression? Same for calculated diffusemaps.

According to iddevnet these also work as fragmentMap parameters.

[Tels]

Use the _flat keyword in this case.

 

Thanx, also added that to my parser (never knewn about it

 

According to iddevnet these also work as fragmentMap parameters.

 

Ah ok. Here is one more special case:

 

tdm_potglass hasn't got a diffuse, instead if got a filter block:

 

	{
blend filter
map models/darkmod/props/textures/potion_glass
}

{
	if ( parm11 > 0 )
	blend	   gl_dst_color, gl_one
	map		 _white
	rgb		 0.40 * parm11
}
{
	if ( parm11 > 0 )
	blend	   add
	map		 models/darkmod/props/textures/airpotion_s
	rgb		 0.15 * parm11
}

 

My code currently extracts a map expression from the TDM frobhightlight as fallback,meaning it ends up with:

 

		map		 models/darkmod/props/textures/airpotion_s

 

aka models/darkmod/props/textures/airpotion_s. Actually, it should be:

 

models/darkmod/props/textures/potion_glass but I am not sure. Extracting the relevant info is a bit hard, one would need to go through all blocks, extract all map expressions and all blend expressions, then look for one that is "filter". Can be done, but is it worth it?

 

Otherwise, the script seems to work fine, I will check it in and then post back here so somebody can try it.

[Baal]

My code currently extracts a map expression from the TDM frobhightlight as fallback,meaning it ends up with:

models/darkmod/props/textures/airpotion_s. Actually, it should be:

models/darkmod/props/textures/potion_glass but I am not sure. Extracting the relevant info is a bit hard, one would need to go through all blocks, extract all map expressions and all blend expressions, then look for one that is "filter". Can be done, but is it worth it?

No, these special cases are better checked by someone anyway and the fallback you're using should be fine.

[Tels]

Okay, the script is checked in. Here is how to use it:

 

* If on windows, install Perl: http://strawberryperl.com/ (5.10.x.x is fine)

* On Linux, should work out of the box

 

* open up a console (or a DOS session on windows)

* change to the Darkmod directory so that "textures", "materials" etc. are the subdirectories

 

* run the script like so:

 

perl devel/frob_hightlight.pl

 

It will complain about a few non-existing textures, as far as I can see that are ones w/o frob-highlighting needed code anyway, so it can probably be ignored.

 

Check the output by doing:

 

* svn diff materials

 

(or the windows/svn tortoise equivalent)

 

If the result is fine, check it inor just tell me to check it in.

 

In case you want your originals back, delete everything in the materials folder and do an SVN update that will restore the original files.

 

Edit: Here is the full diff of the changes generated by the script. It looks like it might be okay, but I haven't done extensive testing:

 

http://bloodgate.com//mirrors/tdm/pub/doc/frob_diff.zip

[Tels]

Hm, the script isn't correct (or something else goofed):

 

* a lot of hightlights look mighty fine, albeit quite too bright (that can surely be tweaked inside frob.vob), like the washboard, some doors etc.

* some objects highlight fine, but it looks like their CM is appearing black overlaid over the object (huh?) when they hightlight, like apples, bread etc.

* some materials look a distinct green like some doors

 

I tested with Chalice from Fidcal and it was very apparent there.

 

Can somebody else please test? Rebb?

[rebb]

Hmmm, could your script maybe put a line like "//::CHECKPLEASE::" inside materials that have no diffuse/spec/normalmap, like that glass you mentioned ?

So then one could do a simple filesearch and change the offending materials manually - since those probably aren't a lot, it should be quite manageable.

 

Going to check stuff using your diff now, thanks .

[rebb]

I think i found whats causing the collision-models to show as black.

 

Your script seems to insert the frob-stage into materials without checking if they had the old frob-stages in them. It would be best if the script detected the old frob-stages, and replaces them with the new one.

 

Thats why the new frob-stage shows up in all the collision-materials too .

[Tels]

I think i found whats causing the collision-models to show as black.

 

Your script seems to insert the frob-stage into materials without checking if they had the old frob-stages in them. It would be best if the script detected the old frob-stages, and replaces them with the new one.

 

Thats why the new frob-stage shows up in all the collision-materials too .

 

Ah, I thought it couldn't hurt that it also appears in materials missing such a stage

 

Tweaking that should be easy, gimme a minute.

[Tels]

Hmmm, could your script maybe put a line like "//::CHECKPLEASE::" inside materials that have no diffuse/spec/normalmap, like that glass you mentioned ?

So then one could do a simple filesearch and change the offending materials manually - since those probably aren't a lot, it should be quite manageable.

 

Going to check stuff using your diff now, thanks .

 

I have added the "// check" comment, but it appears now in 663 places. Ugh. Most of them are actually okay, so should I still leave that in? Or just drop it and we just fix what appears to be broken (instead of checking 663 materials only to find 660 of them are actually right?

[Tels]

Updated with a new diff:

 

http://bloodgate.com//mirrors/tdm/pub/doc/frob_diff.zip

[rebb]

I have added the "// check" comment, but it appears now in 663 places. Ugh. Most of them are actually okay, so should I still leave that in? Or just drop it and we just fix what appears to be broken (instead of checking 663 materials only to find 660 of them are actually right?

 

Hmmm, and those 663 all had the old frob-stages too ? If not, then it's probably enough to only put that check-line into materials that had frob-stages in the first place. 663 seems a bit much, but who knows .

[Tels]

Looks a bit better now in game, except:

 

* things are way too bright for my taste (almost blinded by them in a dark room)

* guards are greenish

* door hinges stay black

* one door and some patch look greenish, too

[Tels]

Hmmm, and those 663 all had the old frob-stages too ? If not, then it's probably enough to only put that check-line into materials that had frob-stages in the first place. 663 seems a bit much, but who knows .

 

Yeah, my code now only inserts the new stage if the old one was there. It also then (andonly then) inserts the comment when:

 

* the diffusemap was not directly defined

* OR the bumpmap is "_flat" (e.g. non-existing)

* OR the specularmap is "_black" (e.g. non-existing)

 

Maybe that logic is a bit too agressive in inserting the comment, but then, I don't know how to decide whether a missing bumpmap is ok, or not

[Baal]

The "please check x" comments are wrong in some materials.

This for example:

diamond_large
{
   noSelfShadow
   noshadows
   twosided
   translucent
   glass

   {   
       blend diffusemap
       map models/darkmod/props/textures/diamond_large_d
       red 0.5
       blue 0.1
       green 0.1
   }


   // TDM Ambient Method Related
{
	blend	add
	map		models/darkmod/props/textures/diamond_large_d
	red		global2
	green	global3
	blue	global4
}

// TDM Frob-Highlighting
// TDM Frob-Highlighting
{
// Please check: bumpmap
if ( parm11 > 0 )
	blend		gl_one, gl_src_color
	program		frob.vfp
	vertexParm	0	0
	fragmentMap	0	_flat
	fragmentMap 1	
	fragmentMap 2	_black
}
}

[Tels]

The "please check x" comments are wrong in some materials.

This for example:

 

Thanx! In this case the diffusemap was not correctly recognized, so the check comment is wrong. However, at least it is there

 

Guess I need to teach the script to hunt around for "blend map" or "blend diffusemap" stages and extract the info from there, too. *sigh* Nothing is ever easy.

 

It is way past midnight here, so I'll do this another day, unless someone just fixes the stuff manually

[rebb]

A missing bumpmap and specularmap is ok, in case of a missing bumpmap the game just uses the vertex-normals - but diffuse is pretty important, otherwise you just have a black lump of geometry, at least on things that are supposed to be opaque.

 

But the main use of the check-line is to know which materials aren't opaque, and that they should be looked at specifically to see what should go into the diffuse slot of the frob.

 

So i'd say it's enough to only insert the check-line when no diffuse is found.

This really seems to be quite the minefield of special cases, argh .

[Tels]

Meh, accidentily deleted by post:

 

A missing bumpmap and specularmap is ok, in case of a missing bumpmap the game just uses the vertex-normals - but diffuse is pretty important, otherwise you just have a black lump of geometry, at least on things that are supposed to be opaque.

 

But the main use of the check-line is to know which materials aren't opaque, and that they should be looked at specifically to see what should go into the diffuse slot of the frob.

 

So i'd say it's enough to only insert the check-line when no diffuse is found.

 

Revised the check again, it now only warns if:

 

* the diffusemap line couldn't be found

* AND we couldn't find a "map blah" line in the old frob highlight code, either

 

That cuts down the warnings to one case and that is actually missing a diffusemap

 

This really seems to be quite the minefield of special cases, argh .

 

You don't want to know how many special cases the parser I am using behind the scenes knows - I am glad that I invested the few hundred hours back then to write it

 

Updated diff on bloodgate.com.

[Baal]

Unfortunately there are also some "blend bumpmap" and "blend specularmap" stages in a few files (~30 files, so I guess they can be checked manually).

[Tels]

Unfortunately there are also some "blend bumpmap" and "blend specularmap" stages in a few files (~30 files, so I guess they can be checked manually).

 

Could you give me an example? Are these the ones that make guards turn green?

 

What now, should I check in the changes and someone tweaks/fixes the rest manually?

[Baal]

There are cases like these:

bc_gembag
{
...
{
     blend bumpmap
     map   heightmap(models/darkmod/props/textures/bc_gembag_h, 6)
   }
   ...
}
}

I believe this is equivalent to "bumpmap heightmap(models/darkmod/props/textures/bc_gembag_h, 6)" so I could rewrite these defs manually before processing them with the script.

 

And these:

textures/darkmod/nature/snow/plain_snow01
{
...
{
       blend bumpmap
       map         textures/darkmod/nature/snow/plain_snow01_local
       scale 0.5, 0.5
   }
...
}

If there are not to many defs that contain bumpmap or specular stages like this I can correct them manually.

 

Don't check in the new materials yet. There are some problems (the green materials) that need to be figured out. I try to run the script myself and see what needs to be done.

[Tels]

There are cases like these:

bc_gembag
{
...
{
     blend bumpmap
     map   heightmap(models/darkmod/props/textures/bc_gembag_h, 6)
   }
   ...
}
}

I believe this is equivalent to "bumpmap heightmap(models/darkmod/props/textures/bc_gembag_h, 6)" so I could rewrite these defs manually before processing them with the script.

 

And these:

textures/darkmod/nature/snow/plain_snow01
{
...
{
       blend bumpmap
       map         textures/darkmod/nature/snow/plain_snow01_local
       scale 0.5, 0.5
   }
...
}

If there are not to many defs that contain bumpmap or specular stages like this I can correct them manually.

 

I could teach my script to figure these cases out. Let me try.

 

However, I am a bit worried about the scale keyword, I don't think that will work with the new hightlight as it doesn't have a scale or individual scales for the individual maps.

 

Don't check in the new materials yet. There are some problems (the green materials) that need to be figured out. I try to run the script myself and see what needs to be done.

 

Cool, please post back

[Tels]

One of the problems is that the script no longer adds the frobhighlight to materials that do not have a frobhigh light already.

 

That is fine for skipping the collision materials, but f.i. the tdm_trees.mtr file has materials w/o fob highlight, meaning you could never have a frobbable tree that highlights.

 

Maybe we should reverse this, add it always, except [insert some condition here, like "is a collision material"]?