Time-Saver Build Fix For SCons/Linux Compilation

[NightStalker 67]

It occurred to me that I should submit this patch now, especially since I see evidence of others (like 'Crinkelite') compiling TDM under Linux.

 

Based on replies to my post in my other recent thread, I realize that it has no chance for 2.05 inclusion, but I want others to have access to it now, so I'm asking that it be added to latest SVN, please.

 

Problem/Fix Description:

 

There is some sort of anomaly with the SCons build system (which is used only for Linux builds, not Windows builds) whereby all items get needlessly rebuilt, every time, when doing TDM builds. In fact, this was encountered by 'NagaHuntress' back in this Aug 2015 post. But, for this and various other reasons while investigating a 64-bit Linux build, she chose to completely switch from SCons to CMake, conveniently side-stepping the problem.

 

The issue fixed by my patch has to do with the SCons (MD5) 'signature' file, which is used to decide when input files used in the build have changed. It's not 100% clear what the cause is, but it may be due to the use of the 'VariantDir()' directives in the 'SConstruct' file.

 

Regardless of the reason, credit for the solution comes from Victor Gaydov's webpage:

Reason

 

The problem occurs because:

• SCons creates individual .sconsign files for source directory and build directory (VariantDir)
• for some reason, SCons writes build metainfo for generated sources to .sconsign under build directory, but searches for it in .sconsign under source directory
• since SCons can't find build metainfo for generated sources, it assumes they should be (re)built, and this happens every time

Solution

 

The workaround is to use single global .sconsign for the whole project. This feature is enabled by setting SConsignFile to an absolute path:

import os.path  
env.SConsignFile(os.path.join(env.Dir('#').abspath, '.sconsign.dblite'))

 

Both Doom3 and, therefore, TDM use a method of specifying the SCons signature file which causes this serious problem.

 

The solution is quite simple, just as Victor stated: specify the signature file using a full path.

 

The attached patch provides this improvement.

 

It was tested under 32-bit Slackware 13.1 running SCons version 2.5.1 and under 32-bit Slackware 14.2 running SCons version 2.4.1, both against latest SVN (6720) and against the TDM 2.04 release.

 

The fix means that now when you're building TDM under Linux, SCons won't (annoyingly!) rebuild the whole tree every time.

 

This change also nicely allows the removal of a hack in the Linux build script file which was used to force the existence of a SCons signature file. Prior to this patch, if SCons was run without a signature file of the expected name already existing, it would report a long error message that began with this:

OSError: [Errno 2] No such file or directory: 'scons.signatures.dblite':

And, worse, it would fail to create the proper signature file ('scons.signatures.dblite'), instead creating an unwanted 'scons.signatures.tmp' file, and thereby causing ongoing grief with every successive run of SCons.

 

This simple patch fixes both problems while also removing the inelegant work-around in the Linux build script file.

 

Please apply to latest SVN.

prevent-scons-from-rebuilding-everything-every-time.patch.txt

[duzenko 654]

I don't have a Linux system but if noone else acts on it I will apply it.

[Crinkelite 9]

I just applied the patch myself and it's working beautifully.

[Hamlet 15]

I applied the patch and compiled on an area which had already everything compiled.

It correctly did not compile anything more. Excellent!

[NightStalker 67]

I applied the patch and compiled on an area which had already everything compiled.

It correctly did not compile anything more. Excellent!

Thanks for the report, Hamlet -- much appreciated. And although I'm a newcomer myself, let me be the 1st to say, "Welcome to the forums!".

 

Now if I could just get someone with SVN checkin privilege to apply this patch, I'd be a happy camper!

[Bikerdude 3740]

Calling Grayman...

[grayman 2967]

I had planned to apply the patch after I tested it.

 

However, the current SVN trunk source doesn't compile on Linux, even w/o the patch.

 

I'm focused on getting 2.05 out the door, but even there the Linux build--though it compiles--causes crashes. My call for Linux people to debug that has gone unanswered.

 

We are unfortunately lacking the Linux devs we used to have to deal with problems like these.

[duzenko 654]

I was going to apply it in trunk w/o testing but I was confused by people in this thread who said they had applied it.

Now I leave it to Grayman.

 

So, umm... Hamlet, Crinkelite, you compiled 2.04, or 2.05, or trunk?

[grayman 2967]

Even though I've been unable to test this, several folks have used it successfully, so I committed it to SVN at 6733.