New tdm_installer and dev builds

[datiswous]

Sure I would say the same for an all contained installation file, but this installer is basically a downloader, it's a total waste to delete everything. It's easy to do yourself if needed anyway.

[thebigh]

If you're updating from 2.09 to 2.10 say, then cancel can only mean to undo everything and go back to how things were. From a new install it could mean either, "I changed my mind, I don't want TDM at all anymore" or "This mirror is glacial. Let me switch to a new one". I suppose both options would be useful.

[datiswous]

Ok, I think it should put the downloaded files into a temp folder it can use when updating again.

[Xolvix]

Seems like the best fix would be to add another checkbox to the "Advanced settings" area on the initial installer screen for the option to keep already downloaded files if the user cancels.

Edited March 12, 2022 by Xolvix

[Oktokolo]

On 3/12/2022 at 9:53 PM, datiswous said:

In fact I just tried installing on a linux laptop. It installed fine till 100% and gave an error that it couldn't open a pk4 file (one of the starter missions). Then it deleted all files, that took the installer 12 minutes to download. I wonder if I should try again...

So what you actually want is corrupted files to just get redownloaded from another mirror instead of cancelling the installation?

[thebigh]

Is this something that happens often enough to be concerned about?

[datiswous]

I posted about my experience and suggestions I had, but it's not worth the effort if nobody else has that problem.

[thebigh]

I don't mean to suggest it's definitely not worth the effort. Just trying to get an idea of how often data corruption does happen. It seems like the kind of thing you could have a checksum to verify. If any individual file fails its checksum, delete only it and try downloading it from another mirror. This would be better than assuming the whole download is kaput and starting from scratch.

 

I guess as TDM gets bigger the odds of something going wrong increase but it's likely to affect mostly new users because those of us who've already been playing will usually only be updating an existing install rather than downloading an entirely new one.

[Oktokolo]

3 hours ago, thebigh said:

I don't mean to suggest it's definitely not worth the effort. Just trying to get an idea of how often data corruption does happen. It seems like the kind of thing you could have a checksum to verify. If any individual file fails its checksum, delete only it and try downloading it from another mirror. This would be better than assuming the whole download is kaput and starting from scratch.

The updater knows the checksums - it scans the installed files and compares checksums to known ones for the desired version to select the files to download. So only replacing the corrupted file and continuing with the downloading is the logical thing to do.

Almost every web server and browser out there also supports resuming of partially downloaded files - so if we had checksums for individual chunks of N MiB in size, only at most N MiB would need to be redownloaded on corruption or disconnect.

Some of TDMs pk4s are hundreds of mega bytes in size and there still are regions (like Australia, China or Germany) where a stable and fast internet connection isn't the norm yet.
I myself only had a slow and shaky wireless "broadband" connection for some months last year. I had to surf the web with images disabled sometimes... Downloading TDM would have taken days (but i would have still tried if i hadn't it installed already).

[stgatilov]

Yes, resumable downloads are not implemented yet.

The main problem is that if you kill tdm_installer with task manager, you will have corrupted data on disk. Trying to use this data on future run will cause unavoidable errors. Aside from that, I'm not sure these __download__XXX.pk4 files are valid zip archives when they are saved, which means that some more code has to be added to support loading them. And yes, leaving all these weird files after cancelled installation might confuse user too.

I think the main problem here is that tdm_installer fails after downloading data.
Normally, this should not happen unless mirrors are screwed, or you have software/hardware problems. And I think it happened very rarely in the past, but there were several reports of such errors lately. I could not reproduce the error, and thus far nobody has even shared a log file of such an error.