What's "on the grid" as far as dmap is concerned?

[SteveL]

Wizards out there, what does "on the grid" mean for dmapping purposes?

 

Background to this question is a post I made in Springheel's "gap" thread re dmapping problems. I did a quick DR script designed to search out and select all worldspawn brushes that were off the grid, meaning to extend it to find slivers and brushes that don't quite touch (=possible leaks). Unfortunately it highlighted a load of brushes cut at angles that were apparently gridsnapped. At first I thought the library code was at fault because I was getting odd results like "grid coordinate 224.0 is not on the 0.125 grid". Turned out that the 224.0 wasn't really 224.0, it was 223.99999999999997 and that when printed to screen by python's str() method, it was getting rounded.

 

When I select the verts of the brush that had that x-coordinate and snap them to grid 16, the offending vertex still has the x-value 223.99999999999997. So it seems there's really no concept of verts being absolutely "snapped to grid", even though all grid sizes are in base 2 and so you'd hope not to have to worry about floating-point errors.

 

All is not lost... Presumably dmap has a tolerance for TINY differences else every map would leak. In tests the other day, I found that it tolerates texture misalignment of up to 2 min grid units (in the texture tool) and will still re-stitch brushes.

 

Does anyone know what tolerance it has for brush alignment?

 

In the meantime, I will try to estimate a good threshold to make the tool useful. I have a script running right now that's churning through a couple of big FMs and spitting out how far all worldspawn brush verts are "off" the 1-grid. I'll tot up the results and I hope to see a huge spike at 0 followed by a trough followed by a second spike at some very small fraction. The idea is, the threshold at which the off-grid detector will kick in is in the middle of the zero part of the chart...

[Moonbo]

This isn't a complete answer, but may help in explaining part of what you're seeing. Right now there is a bug in Darkradiant where sloped brushes will begin to deteriorate (i.e. start morphing off grid) as soon as you build them. The deterioration isn't large but gets worse if you move them around. I believe this happens regardless of whether the brushwork is turned into a FS.

[grayman]

The code recognizes that floating point math can create small errors, by not testing for true equality, but for nearness.

 

So you pick a very small number as an EPSILON, and test things this way:

 

#define EPSILON 0.0001f

if ( abs( firstNumber - secondNumber ) < EPSILON )
{
 // the numbers are equal
}
else
{
 // the numbers aren't equal
}

 

If you apply that concept to your "on grid" tests, things might start looking better.

[SteveL]

I did find another thread where grayman was trying to tackle that problem. Admit I haven't read all 7 pages yet.

 

Like most bright ideas I have for adding something to the mapping process, it turns out that the reason no-one's done it before is that it's not anywhere near as simple as I first thought! The results are in.

Based on a sample of 420k worldspawn verts from a working map, the count of verts that had a given displacement from the 1-grid:

 

< 0.000000000000001  394444 -- all the perfectly-grid-snapped stuff
< 0.00000000000001        0
< 0.0000000000001       142 --\ the floating-point
< 0.000000000001        279 --/ errors
< 0.00000000001           0
< 0.0000000001            0
< 0.000000001             0
< 0.00000001              0
< 0.0000001               0
< 0.000001                7 
< 0.00001               228 --\
< 0.0001               1170 --|
< 0.001                1572 --|  ?????
< 0.01                 4363 --|
< 0.1                  3108 --/
< 1                   16907 -- the 0.125+ grid-snapped stuff

 

Not what I was hoping to see. Yes there's a range where the count is 0, but what is the stuff in the 0.00001-0.01 range? I was hoping to see lots of verts with 0 displacement, and lots in the range 0.1-1 (which is the stuff snapped to the 0.125 grid and above). But there's a whole load of verts with more than the "trough" range that are also technically off grid. Clearly dmap is coping with those (the sample was from In the North by the way -- I thought it apt since grayman had worked on a related problem before). It's still not clear what threshold should be used by a script that's designed to find off-grid stuff.

 

I am going to let the problem rest for now. My brain needs a rest. Time to go make a pretty window

Edited February 3, 2014 by SteveL

[SteveL]

EDIT: cross-posted with you grayman. I'll think about that now... Can you make any sense of my results on your map when it comes to picking a threshold value?

[RJFerret]

In tests the other day, I found that it tolerates texture misalignment of up to 2 min grid units (in the texture tool) and will still re-stitch brushes.

 

I don't believe the texture tool uses grid units (which are 1.1 inches) but much finer control. You'll note if you go all the way until your texture wraps around, it's very conveniently a value of 360. Since textures are applied to surfaces of varying sizes, texture values don't directly relate.

[SteveL]

Another thought: Pehaps the "?????" range is monsterclip and other stuff where you cut freely with the clipper tool and don't worry about grid alignment. I could try re-running the test excluding clip textures... what else should I exclude?

 

@RJ: True, I don't know what the min grid in the texture tool translates to in the normal grid, but the results do back up the theory that dmap has a threshold of some kind.

[grayman]

EDIT: cross-posted with you grayman. I'll think about that now... Can you make any sense of my results on your map when it comes to picking a threshold value?

 

I'm guessing that the points that are measurably off the 1 grid are the result of splitting operations. DR should not be making an effort to keep everything on some grid. The grid is just there for a mapper's convenience, to help him line things up and attempt to reduce the opportunities for gaps.

 

But dmap isn't going to care about a grid. It's just dealing with planes, and where planes intersect to create edges, and where edges intersect to create points.

 

All that intersection math has to deal with the floating point issue, and sometimes it bites you.

[grayman]

... what else should I exclude?

 

Any non-sealing texture.

 

tdm_nodraw*

monsterclip

playerclip

clip

ladder textures

glass

shadow

???

[SteveL]

Thanks, that clarified the problem a bit.

 

I ran the script again, this time counting up textures on the faces that have a vert off the 1-grid in that specific range. The vast majority of it is caulk, monsterclip, and nodrawsolid_stone in that order. Caulk can be sealing or non-sealing of course. Even if I used the same technique over a lot of maps to compile a list of non-sealing textures, it's getting hard to see how this technique could *ever* reduce the false positives enough that it'd be a useful tool :-/ Ho hum.

[RJFerret]

Perhaps leave that to the user?

 

Currently one of the techniques to find internal leaks is to save just the worldspawn to a separate map file, seal up the visportals, and work with that. Obviously those changes would have to be replicated (or merged) with the main map.

 

So I could see doing that, and using your script (if it doesn't pop up every mitered or sloping brush) for insights first. It would be confusing for users who didn't know to perform that routine first though.

[SteveL]

Just the sealing geometry you mean? It already filters to world spawn.

 

I woke up with a few more ideas -- I could import some of DR's filters to ignore clip etc, and distinguish sealing caulk from caulk being used as clip by ignoring faces that aren't near-parallelograms -- but I've also got the feeling that whole approach is misguided. It was based on the observations of many people that something off grid causes glitches. But like grayman said, dmapping has no grid. You might well fix a glitch by moving something back on to grid, but it would have been fixed just as well by moving something else off grid. The point is you need to twiddle something to break a conflict, but the grid isn't the issue, and presumably conflicts in the math can arise from perfectly grid snapped geometry. So instead of being a trivial problem of finding something off grid, this becomes the problem of doing a better job of working out how a map should be dmapped than dmap does itself.

[RJFerret]

Perhaps overreaching functionality now?

 

The original intent seemed to be enabling mappers to find brushes that weren't aligned with their neighbors, since we work on grid, typical problems result from one getting off grid from it's neighbors. (Either protruding in, or not reaching far enough to connect.)

 

It'd be redundant to detect leaks (dmap does that). Overlapping would have value (z-buffer flickering), but plenty of things intentionally overlap (monsterclip).

 

However a "hey, mapper, are these brushes supposed to not be aligned?" would be useful methinks. (Then again, most of my leaks nowadays come from something aligned to the wrong grid line, or a visportal aligned to another that needs to be shifted away.)

[SteveL]

Yeah, I agree what I outlined above would be massive overreach. That was my "I bit off more than I can chew and now I'm putting it down" speech. I think you're right though, the tool could have some uses. It turns out that off-grid was a red herring, and like you say we don't need another leak detector, but it could perhaps become a general script to check for problems that pops up a report like this:

 

Found:

6 slivers

1 duplicated brush

2 nearly-aligned brushes

2 overlapping brushes

 

With a "Select" button next to each item on the report.

 

Clip textures, nodrawsolids, ladders and the like could be ignored by default, by the same method that DR filters them. If people think that'd be useful, it'd be easy enough to write, although what constitutes "nearly aligned" might run into the same problem as "off grid". Last night's script only took an hour -- admittedly, minus some essential testing that should have been done . The problem with it was the concept not the difficulty of writing it, so we could try it out if you want to help test.

[RJFerret]

...so we could try it out if you want to help test.

 

Sounds good, glad too. :-)

[Skaruts]

I found that in a brushwork that I've made and duplicated around an area I got many very thin but visible gaps in game. Apparently it was caused by this issue. At the same time I don't know if it might have made it worse that I had caulk in the adjacent faces between the brushes.

 

How do we work around this? (without having to adjust everything manually and hope for the best)

 

And are there any other concerns as far as making a consistent map goes?

[RJFerret]

The question becomes how were you creating the thin gaps?

 

If you have your grid set to 8, and draw out a wall 8x32, then copy it and drag it 16 over, there's no gap. If you rotate it 90°, and make sure your inside corner vertices are on the same point, there won't be any gap. If you move stuff too far, there'll be a gap, if you don't move it far enough, there'll be overlapping, which causes z-buffer flashing.

 

Yes, manually fix it (it takes less time to not build gaps in the first place--but that takes experience). This topic was more of seeking a scripting solution to discovering such issues in bulk, rather than one at a time when dmap generates an error.

 

Things to watch out for, the boolean subtract tool and the clipper tool used on an angle can lead to things not being aligned on grid. But if you use them on the grid, and don't resize angles, that won't happen.

[Skaruts]

Maybe it isn't this issue after all... I was thinking it was because some gaps like this one seemed to be caused by it, but I found this one that doesn't. It isn't consistent with how the brushes are cut.

 

I avoid small grid sizes by the way. Never liked having to zoom in too much, and I'm aware of their potential dangers. And especially when moving vertices around I prefer to use larger grid sizes, preferably 16 or higher. In this case I think I had to go down to 8 or 4.

Edited February 15, 2014 by Skaruts

[RJFerret]

Those pics remind me of "sparklies", perhaps these threads might relate?

http://forums.thedarkmod.com/topic/13772-mysterious-sparklies-what-are-they-how-to-get-rid-of-them/

http://forums.thedarkmod.com/topic/8910-cause-of-white-sparklies/

 

Two causes given in those: T-intersecting geometry (which it helps to avoid generally) and nearby bright light.