Calling on Linux devs: TDM team needs help

[Bikerdude]

Now that we have a separate branch for 2.05 I would like to enable compiler flags for SSE in trunk and attack any bugs that will come crawling.

My CPU throttles a lot less when running TDM compiled with SSE. Moar fps, you know.

AFAIK, we don't have a working linux package yet.

 

It might be that we'll release the Windows package around Christmas, but have to wait for the linux package to be fixed and released after that.

 

Who's trying to narrow down the linux crash?

I had planned to apply the patch after I tested it.

 

However, the current SVN trunk source doesn't compile on Linux, even w/o the patch.

 

I'm focused on getting 2.05 out the door, but even there the Linux build--though it compiles--causes crashes. My call for Linux people to debug that has gone unanswered.

 

We are unfortunately lacking the Linux devs we used to have to deal with problems like these.

We currently have no Linux devs on the team and we need help getting the linux package tested & fixed for 2.05 so we can release it in the new year.

[NightStalker]

I'm focused on getting 2.05 out the door, but even there the Linux build--though it compiles--causes crashes. My call for Linux people to debug that has gone unanswered.

That's not fair, grayman, and you know it. Both Baal and I have provided all of the information that we possibly can about the crashes we've seen, given the resources you've made available.

 

In fact, I've gone beyond that, bisecting the source code backwards from the latest SVN and building TDM many times, to try to identify the change(s) that cause the crash. But I gave up in frustration (and exhaustion) after finding a reference point (SVN #6642) that successfully compiled but, sadly, failed to actually run a mission. Since I obviously didn't make any of those SVN commits between 2.04 and 2.05 (let alone between 2.04 and latest SVN), I'm not best qualified to determine what broke the build. And I stated as much in my post about all this.

 

Grayman, you've asked for help but you seem to expect people to do so with one hand tied behind their back! How am I to properly debug a Linux build, made by you, with an outdated compiler, with symbols stripped on 1 of the 2 binary files, with no source code available?!? When Baal asked (on Dec 8th) for the 2.05-beta source code, which is something I was 1 breath away from doing myself, he's told to "PM taaaki". Both he and I PMed taaaki for access to the source code, myself on 10 Dec 2016. Today, 9 days later, I still have not gotten a reply to that PM. When you posted the URL to the 2.05-beta SVN tree, both Baal and I found that it requires a login and password. Baal mentioned that, so I didn't repeat it. When you asked yesterday in the beta-test thread "Who is attempting to run on linux, but can't?", I mentioned that I could not access the source code without login+password. You (typically) said nothing. How do you expect anyone to debug 2.05-beta under Linux when we're thusly hog-tied?!?!?

 

Grayman, you've been decidedly less-than-helpful and quite terse from the very 1st post I made on this forum. I've tried to ignore it and work past that, but for you to now say things like "My call for Linux people to debug that has gone unanswered." is demotivating and, therefore, counter-productive.

 

So, grayman, I'm still willing to work with you (and any/all others) if you'll agree to be more accommodating, less terse, and generally more helpful overall.

[nbohr1more]

I think that was more directed internally at our own Dev team (cough, Taaaki)?

[Bikerdude]

• That's not fair, grayman, and you know it. Both Baal and I have provided all of the information that we possibly can about the crashes we've seen, given the resources you've made available.
• How do you expect anyone to debug 2.05-beta under Linux when we're thusly hog-tied?!?!?
• Grayman, you've been decidedly less-than-helpful and quite terse from the very 1st post I made on this forum. I've tried to ignore it and work past that, but for you to now say things like "My call for Linux people to debug that has gone unanswered." is demotivating and, therefore, counter-productive.
• So, grayman, I'm still willing to work with you (and any/all others) if you'll agree to be more accommodating, less terse, and generally more helpful overall.

I will have to stop you there fella,

 

The main and I suspect the only reason Grayman hasn't gotten back to you is because he is extremley busy, what little time he has that can be devoted to the mod and in general is very limited. Grayman is one of the most productive members of the team, so I can assure you he isn't going out of his way to antagonise - to be blunt he dosen't actually have the time.

 

So talking the above onboard, lets take a deep breath and a step back. In the meantime we have asked Taaaki to grant you SVN access, and we can take things from there etc.

 

Lastly the team always tried to fix things internally, but on this occasion some of the comms was relevant to the request @Gman - apologies if this wasn't the case.

 

b.

[grayman]

O-kaaay. I'll admit to not watching carefully, because there are at least 3 people involved, with various linux systems and multiple "try this, then that" suggestions. I figured the discussion among those folks would eventually lead to a solution. There's not much I can do to help that along, so my time has been focused elsewhere.

 

I'll PM taaaki to hook you into the SVN source code.

 

I'm stuck with the so-called "outdated" linux components because linux moves too fast for me to constantly be updating it and proving it builds on my virtual machine, and having some guarantee that it's going to run on the plethora of linux machines our TDM players use. I can't run TDM on my linux virtual machine. One previous attempt to upgrade to Ubuntu 12-something was a nightmare, and the resulting binaries wouldn't run for our linux people, so I decided at that point to just stick with what had worked in the past.

 

The source code for the original 2.05 package build was taken at SVN rev. 6684.

 

Subsequent builds included these merges:

 

11/26 - 6690-6694

12/2 - 6698, 6707-6710

12/11 - 6718, 6720

 

I don't want folks committing to the 2.05 source branch, so when you get access to the sources, please don't do that.

[duzenko]

In fact, I've gone beyond that, bisecting the source code backwards from the latest SVN and building TDM many times, to try to identify the change(s) that cause the crash. But I gave up in frustration (and exhaustion) after finding a reference point (SVN #6642) that successfully compiled but, sadly, failed to actually run a mission. Since I obviously didn't make any of those SVN commits between 2.04 and 2.05 (let alone between 2.04 and latest SVN), I'm not best qualified to determine what broke the build. And I stated as much in my post about all this.

 

When Baal asked (on Dec 8th) for the 2.05-beta source code, which is something I was 1 breath away from doing myself, he's told to "PM taaaki". Both he and I PMed taaaki for access to the source code, myself on 10 Dec 2016. Today, 9 days later, I still have not gotten a reply to that PM.

I'm still waiting Taaki to answer my month-old request It's not like he ignores you only.

#6642 crashes for you - how about #6640 or #6637?

I could probably upload a particular svn branch if you aren't granted access in the next hours.

P.S. I really appreciate your efforts, and I admire the "do it now" spirit a lot. AFAIR Grayman warned that he has really little time this time of year.

[Bikerdude]

@Duzenko, like Gman - Taaaki is also very busy

 

Calling Taaaki...

[grayman]

So, grayman, I'm still willing to work with you (and any/all others) if you'll agree to be more accommodating, less terse, and generally more helpful overall.

 

My apologies. The 70hrs/week I put in for several years on this game have dwindled to less than 10 or so due to sometimes insurmountable RL issues. These require that I ignore a ton of stuff I used to help out with in the old days.

 

As I've said for the past couple years, I'll be happy to unload the package release work to anyone who steps up. No one has.

 

I'd be quite happy just fixing occasional bugs and working on missions.

[duzenko]

As for the memcpy crash.

There is one memcpy in the code I contributed:

void idRenderSystemLocal::CaptureRenderToBuffer(unsigned char* buffer)

Does it crash if you toggle the

if (1)

in there?

[nbohr1more]

I wonder if that should be controlled via a cvar boolean? Currently it's a pretty strange conditional...

[duzenko]

I did not put cvar because I did not expect any blowback and did not have access to svn

Still need to confirm that's it.

Not sure if cvar is optimal solution if it works on windows and not on linux - ifdef?

The if (1) is just a toggle-able alternative to //

I used it in a couple more places since then.

[Bikerdude]

@Linux guys, if the above is related to the "Compiling TDM 2.04 Under Linux" thread can I move the relevent posts there..?

[duzenko]

Parts of this discussion are not really technical IMHO and would be irrelevant flood in there

[NightStalker]

All these replies are coming faster than I can answer.

Let me be clear: I have no beef with taaaki or his delayed reply. I've asked a favor of him (access to 2.05-beta source code), not the other way around. So I have no expectations of him. If he's busy, that's OK (with me, anyway).

Up to now, I've even been OK ignoring many of the things I complained about above. But when grayman said, publicly, that "My call for Linux people to debug that has gone unanswered.", that was simply unfair and more than I could ignore.

Grayman, I truly respect both your time and commitment to this project. Since I haven't said so before, I'll do so now: Many, many thanks for your efforts! I genuinely appreciate it. That goes to you and everyone else who's contributed.

The source code for the original 2.05 package build was taken at SVN rev. 6684.

Subsequent builds included these merges:

I wish all this information had been public 2 weeks ago, when I'd have had more time to put it to good use. I still plan to test and try to help based on this new information, but my time is quite limited now (but may improve in a week or so). Nevertheless, thank you for making that clear -- it will assist me immensely and I'll report more as soon as practical.

I can't run TDM on my linux virtual machine.

That's also good to know. (I was assuming otherwise.) FWIW, I'll cut you way more slack in that area now.

In the meantime we have asked Taaaki to grant you SVN access, [...]

I'll PM taaaki to hook you into the SVN source code.

I don't want folks committing to the 2.05 source branch, so when you get access to the sources, please don't do that.

My thanks to both of you.

However, although I would accept and be exceedingly careful about SVN access if granted, it's not an immediate need. I certainly have no intention of committing anything to SVN at this point (except possibly that SCons/Linux fix, tested by 3 people). Someone (preferably the one that broke things) needs to fix the things that broke the Linux build. I will gladly test anything that's asked of me, time permitting, to assist in this effort. I may even initiate tests of my own doing, given this new information from grayman. But I have no immediate plans to alter any code in SVN until after the time when I can build and run latest SVN, and even then certainly not without publicly running it all by the dev team first. My big problem is that I can only compile under Linux and not under Windows. Frankly, that would make me very nervous about committing any non-trivial changes to SVN (now or ever).

As I've said for the past couple years, I'll be happy to unload the package release work to anyone who steps up. No one has.

I'm not stepping up, but I may be willing and able to help in that regard. Frankly, the current build system is "in need of some love" and I've got ideas. That SCons/Linux patch was just a simple starting point. More on that much later, after all this 2.05 stuff settles, and in some other thread.

I have to ask... What's this burning need for a deadline here? If it's broken, let's take the time to fix it, properly.

@duzenko: I'll try all the stuff you suggest and report back, but I feel that such talk belongs in a different thread, so I may report elsewhere. (Bikerdude, you read my mind about moving those comments elsewhere.)

[Bikerdude]

It sounds like it was a good idea for me to put this out there then

 

Hopefully between Nightstalker and Duzenko, we will have the linux side of things in good hands.

[nbohr1more]

Heh, looks like you are a pretty sturdy chap. Thanks for your persistence.

 

If you ever delve into the history of the project, you'll see that coming from TTLG we've had a history of being a bunch of grumpy malcontents

who angrily rebuked Thief: Deadly Shadows and then moved on to angrily argue with each other !!!

 

If some of the old devs were around, assuredly there would be some sorta hate fueled arguments in CAPS right now...

[NightStalker]

It sounds like it was a good idea for me to put this out there in the ['open', I presume] then

Yes, IMHO, it's a very good idea and I thank you for doing so. Open, public communication, however heated it might occasionally get, is vital to a community project like this one.

If you ever delve into the history of the project, you'll see that coming from TTLG we've had a history of being a bunch of grumpy malcontents who angrily rebuked Thief: Deadly Shadows and then moved on to angrily argue with each other !!!

Well, I've never been accused of being shy or afraid of confrontation. So I suspect I'll fit in well. In all seriousness, though, I'm usually quite easy to get along with.

[freyk]

Can someone take also a look at my binary installation packages? https://freyk.wordpress.com/tdm-installer/

 

64 bit:

http://forums.thedarkmod.com/topic/18446-tdm-doesnt-start-on-ubuntu-1604/

Edited December 20, 2016 by freyk

[NightStalker]

Freyk, I seem to recall that your GUI installer app calls the existing (command-line) 'tdm_update{.linux|.macosx}' installer app, which is itself in need of at least 2 fixes, IMHO. I have plans to look into that and patch some fixes, but, for the overall good of the TDM project, I'm much more interested in getting 2.05-beta code working now than I am in any installer issues.

So, speaking only for myself here, simply due to time limitations (i.e. rather than a lack of interest), I won't be testing your installer until (at least) the existing 'tdm_update' installer is fixed to my satisfaction. Sorry.

As for the 64-bit instructions you linked to, for TDM, I run neither Ubuntu nor a 64-bit OS, so I can offer no advice there.