DR and its shifting planes

[grayman 2687]

It's taking hours to revert these libraries back to Jan 2011.

 

Hope I don't have to do this too often.

[Tels 267]

If the old revisions have been inSVN (DR was converted to git rather recent, wasn't it?) you can also use SVN diff and SVN blame to hunt down specific changes.

[grayman 2687]

Okay, so I took several hours to revert DR and w32deps to 1.5.1, and tested it to make sure it wasn't shifting brushes.

 

It wasn't.

 

Am I supposed to revert everything back to the current version in order to use bisect? Everything from 1.5.1 is no longer in the snapshot of the repository I currently have, so I can't use bisect on it.

 

And if so, once I've done that, and I give bisect the boundaries, does bisect revert the repository to the version midpoint between the good and bad markers?

 

If so, I expect that each subsequent bisect of the repository is going to take hours as it reverts back and forth through time, not to mention that I have to do w32deps by hand to match the moment in time bisect is currently using.

 

Btw, w64deps did not revert back to 1.5.1 correctly. I got pages of "tree conflict" errors, rendering the result useless, so I can't test 64 bit.

[grayman 2687]

I may be able to do some testing in Linux too if the problem is easy to reproduce — are you able to make a simple test map that demonstrates the problem, perhaps with a single brush that has some unusual plane orientations, and then commit this into the maps/test/dr directory in the mod asset repository?

 

I'm beginning to sense that my foray into DR is coming to an end, given the amount of time needed to reproduce older versions and isolate the bug, so In SVN TDM rev 13280, I committed two map files showing the warping of 2 brushes in a func_static over a series of writes/reads.

 

maps/test/dr/brushShift1.map (the original file)

maps/test/dr/brushShift2.map (the warped file)

 

I'll give this one more day. If I can make any progress, fine. If it turns into "wait 6 hours, test something, wait 6 more hours, test something, etc." I'll have to abandon the effort and wait for someone who's familiar with DR to find and fix the problem.

[nbohr1more 1883]

Possible relevant trackers in 1.6.0 (the next most likely culprit):

 

2653

2610

2713

[grayman 2687]

You use the hash tag itself, e.g.

 

$ git bisect start

# Set the HEAD (latest) revision as the bad commit
$ git bisect bad HEAD

# Set the last-known-good commit
$ git bisect good c1cb017729c4364ad1dc3886845663ffeb358b35

 

I restored the repository to its current revision.

 

However, when I try to kick off a bisect, I get this on the "git bisect good ..." line:

 

fatal: needed a single revision

Bad rev input: c1cb017729c4364ad1dc3886845663ffeb358b35

[OrbWeaver 448]

Am I supposed to revert everything back to the current version in order to use bisect? Everything from 1.5.1 is no longer in the snapshot of the repository I currently have, so I can't use bisect on it.

And if so, once I've done that, and I give bisect the boundaries, does bisect revert the repository to the version midpoint between the good and bad markers?

 

Bisect automatically switches the Git-controlled source to the correct half-way point, which should not itself be a slow operation, but yes you would potentially need to manually revert the dependencies by hand because these are not tracked in Git.

 

I'll give this one more day. If I can make any progress, fine. If it turns into "wait 6 hours, test something, wait 6 more hours, test something, etc." I'll have to abandon the effort and wait for someone who's familiar with DR to find and fix the problem.

 

That's fair enough given how much time it's clearly going to need to do this on Windows. I will have a crack at doing the bisect on Linux and hope that there aren't too any insurmountable build problems with the intermediate versions.

 

At least you have determined that the problem was introduced rather than being a fundamental issue with the maths, which is a huge step forward given that fixing the actual maths would probably be beyond my expertise.

 

fatal: needed a single revision

Bad rev input: c1cb017729c4364ad1dc3886845663ffeb358b35

 

The commit hash in my post was just a random example chosen from a Git repository I have here at work, not a specific revision in DarkRadiant that you should use. I suspect it's complaining because that hash doesn't actually exist in the DR repository.

[grayman 2687]

That hash tag is valid, from the commit on 1/22/2011 marked "Bump version to 1.5.1".

 

Edit: Please look at the git history and tell me what to type to mark that spot as "good".

[grayman 2687]

That's fair enough given how much time it's clearly going to need to do this on Windows.

 

Well, Windows isn't the time-sink in this case.

 

The time-sink is the miserable 4kB/s transfer rate I'm getting from Sourceforge as I'm trying to place w32deps at the right point in time.

[OrbWeaver 448]

Edit: Please look at the git history and tell me what to type to mark that spot as "good".

 

This is the exact sequence of commands I use on Linux, along with git's output:

 

$ git bisect start
$ git bisect bad master
$ git bisect good c1cb017729c4364ad1dc3886845663ffeb358b35
Bisecting: 452 revisions left to test after this (roughly 9 steps)
[1419155347efb8ba405a0c934c0b7ebccf5b3436] Fix skin preview, better default viewing angle.

[grayman 2687]

Well, this time the bisect worked.

 

Unfortunately, as expected, the w32deps library is not in sync with the version selected by bisect, and the build output is littered with errors.

 

-----------------------------

 

At this point, I'm abandoning this experiment.

 

OrbWeaver, if you can find the time, please see if you can squash this bug. I've filed it in bugtracker as issue #3272, and I've submitted a couple map files showing the problem.

 

Thanks for your help.

[Tels 267]

Well, Windows isn't the time-sink in this case.

 

The time-sink is the miserable 4kB/s transfer rate I'm getting from Sourceforge as I'm trying to place w32deps at the right point in time.

 

If a faster server is a solution, would it help if the git repository is cloned on a faster machine? How big is the entire repository?

[grayman 2687]

My "slow xfer speed" comment was wrt the w32deps and w64deps repositories on Sourceforge. They're still under SVN, and haven't been moved to GIT.

 

I had no speed issues with retrieving from DR's GIT repository. I don't know if that's on Sourceforge or elsewhere.

[Tels 267]

If it is this one:

 

svn co https://darkradiant.svn.sourceforge.net/svnroot/darkradiant/trunk/w32deps .

 

I get a full speed as fast as my pipe allows. But cloning the entire repository would need some massive storage space and bandwith, so its probably much better if someone zips up the entire repository on the server and provides a download of that (not sure that this can be done one sourceforge, tho).

[OrbWeaver 448]

At this point, I'm abandoning this experiment.

 

Although I could do the bisect, I too had build problems on Linux so I conclude that bisect is not going to work for a commit range of this magnitude (it is very useful for finding a bug in some recent commits, but this is several years of development). It will have to be regular debugging in this case.

 

OrbWeaver, if you can find the time, please see if you can squash this bug.

 

I can't promise a solution but I am willing to have a look at it. It reproduces very easily with your test map, so it should hopefully be a fairly straightforward process of tracking the particular values through the load/save pipeline and finding where the difference appears from, and then comparing this to whatever the previous code did before the bug was introduced.

[grayman 2687]

Thanks for looking at it.

 

Sotha, Biker, and myself have pointed out that this is very deflating, having to redo work that we thought was already nailed down.

 

We all look forward to a solution.

[Bikerdude 3736]

Sotha, Biker, and myself have pointed out that this is very deflating,

We all look forward to a solution.

+1 on that..

[OrbWeaver 448]

Am I correct in thinking that this behaviour only applies to non-worldspawn brushes (i.e. func_statics and other models)?

[New Horizon 470]

Am I correct in thinking that this behaviour only applies to non-worldspawn brushes (i.e. func_statics and other models)?

 

I thought it 'only' applied to worldspawn and that func_statics and models were safe.

[Obsttorte 1440]

This behaviour thos apply to both worldspawn and func_static

 

But as far as I recognized it in my own map files it's a bit random

[OrbWeaver 448]

This behaviour thos apply to both worldspawn and func_static

 

Are you absolutely sure that it also affects worldspawn? The test map Grayman provided uses a func_static at a very high distance from the origin, and I have identified a problem whereby the map saving code is actually modifying all of the brushes before export in order to subtract the func_static's origin (presumably to make the brushes relative to the func_static instead of the world origin), and then adding it back afterwards, which results in a small change to the value (in the same direction) each time you hit the save button.

 

However this code path explicitly avoids changing worldspawn brushes, so if worldspawn is also affected there must be another problem.

[Obsttorte 1440]

Well, at least I thought so, as I barely remember to see such behaviour on some early maps, but I can be wrong, of course

[Sotha 1905]

I'm testing now...

 

EDIT:

I took a piece of detail that was corrupted. I snapped it back to grid and put it into a separate map.

I cloned it and made the other f_s and other worldspawn. Then I artificially 'aged' the mission by loading it, adding some other brushes, save, load again for tens of cycles.

 

Here it is:

The left one is worldspawn and the right one is f_s. The f_s is corrupted, but wordspawn is as beautiful as the original!

[grayman 2687]

I've gone through Home Again and I can find a couple worldspawn brushes that look like they've shifted off grid. But this might be the result of clipping. They were done so long ago I can't remember. I'm not seeing multiple blatant accumulated errors like I do with func_statics.

 

I'm going through the final map for In the North. Back in a bit.

[OrbWeaver 448]

Non-worldspawn brushes only would certainly be consistent with earlier posts in this thread about the problem occurring on detail work, as well as the fact that users aren't reporting world leaks by the dozen, which would surely be expected if worldspawn was being corrupted.

 

However the code to add and subtract the origin seems to have been there since 2007, so it's not obvious where the actual defect appeared. The problem itself may not be that the geometry is being modified with every save (although this seems risky to me), but perhaps some particular way the calculation is being done results in these accumulating errors.