Stalls on Install

[Caverchaz]

I got tdm_installer.exe but when I run it, it eventually stalls on some pk4. file. I try and try again and it stalls and stalls again. Can anyone assist?

Thanks

Ed

[stgatilov]

2 hours ago, Caverchaz said:

I got tdm_installer.exe but when I run it, it eventually stalls on some pk4. file. I try and try again and it stalls and stalls again. Can anyone assist?

When does it happen?
On the first page?

Maybe it is just some large pk4 file... How long did you wait?

Could you find file named like "tdm_installer_1621300601.log" near the installer and share it here?
I guess the latest one should be enough.

[Caverchaz]

The install is on the DarkMod installer V1.03 (built on Mar 15 2021). The text box shows the current file being downloaded and it has differed each time. I have over 40GB on my machine. Twice, I let the process run overnight and it was in the same spot the following morning.

I have attached the install log file. 

Thank you so much! 

tdm_installer_1621295418.log

[stgatilov]

What type of internet connection do you have?
What is typical download speed?
Do you have something which uses network constantly... like torrent?

The downloading always hangs on the same mirror.
Of course, we don't have too much mirrors yet (perhaps time to change that).
But out of 20 attempts it is always the same mirror.
@cabalistic, maybe you could look at something there?

[cabalistic]

22 minutes ago, stgatilov said:

@cabalistic, maybe you could look at something there?

Can't find anything in the logs. Mirror itself is up and running, I've been downloading from it regularly with the installer without any issues.

Does the installer have any kind of error handling and reporting for connection issues? I.e. would it notice if the connection is closed or unresponsive, or if an error code were returned? (Although I should have seen one in the logs, but you never know...)

[stgatilov]

You can try this way as a workaround:

1) Open tdm_installer.ini, and find there the following lines:

url_2=http://tdm.frydrych.org/mirror/zipsync
weight_2=300

Add sharp character # at the beginning of both of these lines.

2) Try to use tdm_installer again, but check Advanced Settings and Skip config file download on the first page.

This might help to download the game for the first time. If it helps, don't use it ever again: most likely you won't have this issue when switching between versions.

[stgatilov]

3 minutes ago, cabalistic said:

Does the installer have any kind of error handling and reporting for connection issues? I.e. would it notice if the connection is closed or unresponsive, or if an error code were returned? (Although I should have seen one in the logs, but you never know...)

I think it depends.

If HTTP error happens, then indeed it will stop and show error message. If connection is closed, then I hope it will detect it too. However, I don't think there is any protection from server silently falling to sleep, no protection against server not sending data or sending it too slow. Maybe some timeout somewhere stops connection in such a way that libcurl does not detect it.

The good approach would be to set upper limit on size of one download request, set some lower limit on download speed, and retry (perhaps with progressively smaller chunks and softer limits). But I think I need to at least understand how it happens and be able to reproduce it. Any ideas?

[stgatilov]

Just to rule out more stupid possibilities and be sure it is actual downloading.

@Caverchaz, could you please try to fresh-install TDM once again without any tweaks, wait until it hangs reliably, then open task manager, find tdm_installer.exe, Right+click on it and choose "create dump file" or something similar. Then find the file, 7zip it and share via some cloud storage.

This wiki article has detailed instructions, but for the case when DarkRadiant crashes --- while we need to apply it to tdm_installer when it hangs.

[Caverchaz]

@stgatilov Altering  tdm_installer.ini as you indicated worked. Thank you so much. I will create the Dump File and share it via cloud storage.

Thanks again!!!!!!

[cabalistic]

4 hours ago, stgatilov said:

Any ideas?

At the moment, only one. We could try to test a download from my mirror with artificially limited download speed. Perhaps there's a hidden timeout somewhere if download takes too long, although I didn't find anything obvious.

[stgatilov]

@Caverchaz, I have added some retry capabilities to the installer.
Ideally, it should be able to overcome the problem you had.

Unfortunately, I cannot test it myself: I installed TDM afresh without any issue yesterday.
The problem simply does not happen on my machine. But you have somewhat unique conditions that the problem happens on your side.
So I'd like to ask you for help with testing.

Here is the plan:

1. Put tdm_installer into empty directory and run without any checkboxes set (just as you did initially when it hung up).
2. If it hangs (does not show any progress for ~10 minutes, or simply does not finish in a few hours), then take a memory dump as I described above.
3. Regardless of whether it succeeds or hangs, please find the logfile of the run and attach it here.

 

[Alberto Salvia Novella]

HANGS AT MANIFEST

For me running the installer with "--unattended" result in a hang from time to time at:

    Downloading "http://tdm.frydrych.org/mirror/zipsync/release/release200/manifest.iniz"...

It seems to depend on how busy the server is at that moment. After many tries it couldn't work at all, but if you try later it could always work.

 

OTHER FLAWS

Editing the .ini file doesn't work. The installer overwrites it before starting the download.

Also closing the installer window doesn't interrupt the download process, only the window. The terminal still prints as downloading.

 

NEEDS TO TIMEOUT

I would check in the code that the connection times-out after a second if it doesn't receive a response. Then it should retry it, for example, 3 times after 30 seconds plus a random number from 0 to 10.

For example when I use "curl" inside my code, I always do like this:

    curl --silent --connect-timeout 1 --retry 3

memory_info.txt

Edited May 24, 2021 by Alberto Salvia Novella

[cabalistic]

5 hours ago, Alberto Salvia Novella said:

It seems to depend on how busy the server is at that moment. After many tries it couldn't work at all, but if you try later it could always work.

Unfortunately, the server isn't even close to being busy, so there has to be another cause. But I'd need a concrete timestamp (including timezone) and the client IP for a failing request to investigate further. Perhaps a traceroute to tdm.frydrych.org at the time of failure could also help to rule out issues.

[Alberto Salvia Novella]

The traceroute fails to continue at:

    13  netcup-gw.bbr01.anx84.nue.de.anexia-it.net (144.208.211.31)

Around 17:25 UTC.

This looks as a double issue to me. The server is unreachable, and the installed cannot tell.

Edited May 24, 2021 by Alberto Salvia Novella

[cabalistic]

That means that the request never even reaches my server. I can ask my ISP about it as that's their infrastructure, but I'd still need a source IP to help them identify the incoming request.

[stgatilov]

15 hours ago, Alberto Salvia Novella said:

For me running the installer with "--unattended" result in a hang from time to time at:

    Downloading "http://tdm.frydrych.org/mirror/zipsync/release/release200/manifest.iniz"...

It seems to depend on how busy the server is at that moment. After many tries it couldn't work at all, but if you try later it could always work.

Network is an interesting thing: if something can possibly fail about it, then it will surely fail for someone sooner or later.

I cannot believe it actually hangs, since default CURL behavior is 300 seconds timeout on establishing connection.
I think it can be lowered to e.g. 20 seconds.

 

Quote

I would check in the code that the connection times-out after a second if it doesn't receive a response.

Where did you find it?

 

Quote

Editing the .ini file doesn't work. The installer overwrites it before starting the download.

Installer always does that. It should not be a problem.
 

Quote

Also closing the installer window doesn't interrupt the download process, only the window. The terminal still prints as downloading.

Yes, closing the window does not terminate program.

I think I should tie closing GUI to canceling download in the last page, and simply block the cross icon at all the other moments.

 

 

[Alberto Salvia Novella]

16 hours ago, cabalistic said:

I can ask my ISP about it as that's their infrastructure

Yes, but still this is an issue in the installer too. The installer must expect the network to fail from time to time. It remembers me of this.

8 hours ago, stgatilov said:

default CURL behavior is 300 seconds timeout on establishing connection.

I tell you warranted that, at least when using the curl command, if I don't set a timeout to certain servers the connection will hang forever in many situations.

For example GitHub will hang the connection from time to time to prevent bots from retrieving repository info without using the API. But if I set a timeout and a retry it will always work.

8 hours ago, stgatilov said:

Where did you find it?

Find what?

8 hours ago, stgatilov said:

Installer always does that.

Then anything you change on the ".ini" won't take effect, will it?

8 hours ago, stgatilov said:

I think I should tie closing GUI to canceling download in the last page, and simply block the cross icon at all the other moments.

The least surprising behavior would be that you can close the window at any time. A good enough solution would be that the sub-process continues downloading the current file, and after that it doesn't continue anymore:

while processIsAlive(parent) && file < len(files) {

    downloadFile (files[file])

    file++

}

 

Edited May 25, 2021 by Alberto Salvia Novella

[Alberto Salvia Novella]

I have discovered that effectively the "curl" command will hang, even with those timeout settings.