Jump to content
The Dark Mod Forums

Unexpected CURL error 18


myfriendstiiv

Recommended Posts

thanks, here's the full message

ERROR: Unexpected CURL error 18 on URL http://darkmod-alt02.taaaki.za.net/release/zipsync/release/release200/fms/saintlucia/saintlucia.pk4

I'm using the lastest tdm_installer and it starts to install some of the files but then stops the install with this error.

I don't see an option to change the mirror

 

  • Like 1
Link to comment
Share on other sites

3 hours ago, myfriendstiiv said:

I don't see an option to change the mirror

You can check "Custom version" on the first page, then select mission in dropdown box on the second page.

The error 18 means:

CURLE_PARTIAL_FILE (18)

A file transfer was shorter or larger than expected. This happens when the server first reports an expected transfer size, and then delivers data that doesn't match the previously given size. 

Sounds like a server bug...

I think we should watch over similar reports. If the same mirror gives the problem all the time, then perhaps disable it.

Link to comment
Share on other sites

thank you, so I checked "get custom version" and tried all of the mirrors only to receive the same CURL error 18,

I tried closing the installer down and restarting but still get the same error, only sometimes it was displaying a slightlly different URL

 

   

Edited by myfriendstiiv
removed quote
Link to comment
Share on other sites

3 hours ago, myfriendstiiv said:

I checked "get custom version" and tried all of the mirrors only to reveive the same CURL error 18, I tried closing the installer down and restarting but still get the same error, only sometimes it was displaying a slightlly different URL  

Could you please do a small test using curl executable?

You can download it here, unpack archive somewhere.
Then find where curl.exe is located, and put the attached .cmd script into the same directory.
Then run the .cmd file.
After that, the files named curldata, curlerr, and curlout (no extension) should be created in the same directory.
Send them back, i.e. attach to forum post.


What kind of connection do you have?
Is there some kind of limitations?
 

I can provide raw files for download somewhere to help with initial install.
But when you need to do update, the same issue will appear...

tdm_curl_test.cmd

Link to comment
Share on other sites

6 hours ago, myfriendstiiv said:

thanks for your help,

I have uploaded new version of installer, which writes all performed requests to logfile.

Could you please retry installation once again?
The installer will update itself automatically to version 1.07.

After installation fails, find the latest logfile named like tdm_installer_1625538036.log and attach it here.
 

Link to comment
Share on other sites

1 hour ago, myfriendstiiv said:

the installer did update, but it seems to have failed with the same CURL error 18, heres the two log files

Could you please try a new cmd script in the directory with curl.exe?
It executes exactly the request by tdm_installer that fails...
Share back curldata2.bin, curlhead2.txt, curlout2.txt, curlerr2.txt.

 

tdm_curl_test2.cmd

Link to comment
Share on other sites

1 hour ago, myfriendstiiv said:

done

Ok, we have managed to reproduce the problem outside of tdm_installer.

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0 5514k    0  4067    0     0  18613      0  0:05:03 --:--:--  0:05:03 18741
curl: (18) transfer closed with 5642980 bytes remaining to read

Looks like it read miniscule amount of data, then hanged for 5 minutes, and finally stopped with error.

The 4067 bytes that it managed to get are correct, but I have no idea why it stopped there...
The HTTP headers are also good, exactly as I get on my side.

 

Maybe you could try different variations of the same script and see which things work and which don't?

In the good outcome, you should not get the error, and curldata2.bin should have size 5647047 bytes.
If there is an error or .bin file is smaller than necessary, then it failed.

You can edit .cmd file with any text editor.
Here are things to try:

  1. Replace "http://darkmod-alt02.taaaki.za.net/release/zipsync" prefix in URL with "http://darkmod.taaaki.za.net/release/zipsync"   or "http://tdm.frydrych.org/mirror/zipsync ", "http://mirror.mstrmnds.me/releases/zipsync" . All these servers store the same data. If you are not sure whether URL is correct or not, you can copy/paste it to browser, and it should initiate file download if URL is right.
  2. Remove additional parameters "-Y 1000 -y 10 --connect-timeout 10". Although it is very unlikely it will change anything.

Also I think it's better to try each configuration 2-3 times, in case one request is dropped inconsistently.

 

@taaaki, maybe you can check some logs on the mirror?
Is some information necessary to identify the particular user's requests?


Right now it looks like either @myfriendstiiv has some issues with connection (OS, router, provider, or country messes with traffic), or the combination of particular user and particular mirror gives some extra issues.

I'm not sure what to do.
Since recently I have added retries with progressively lower limits, including retries with single byterange only, but it only triggers only on timeout. Maybe I should just give up and do these retries unconditionally, i.e. on ANY type of error including even malformed data and wrong size.

Link to comment
Share on other sites

I managed to use the attached curl logs to find some of the relevant entries in the logs and correlate those back to a source IP. All the request from the source IP return 206 status (partial content success) and further checks on just the name of the file that was failing also return 206 (or 200 in the case of direct browser downloads). Nothing out of the ordinary in the web server error logs (other than failed requests for FMs that aren't on the server).

The darkmod-alt02 server isn't running a firewall, so it can't be that. And am I correct in understanding that this isn't just happening with this one server or is it isolated to darkmod-alt02?

I am the bat. The night is mine.

Link to comment
Share on other sites

36 minutes ago, taaaki said:

All the request from the source IP return 206 status (partial content success) and further checks on just the name of the file that was failing also return 206 (or 200 in the case of direct browser downloads). Nothing out of the ordinary in the web server error logs (other than failed requests for FMs that aren't on the server).

I guess we are only interested in multipart byterange requests, which are failing at the end of the log. These should return 206.
Like this one:

curl http://darkmod-alt02.taaaki.za.net/release/zipsync/release/release202_from_release201/tdm_ai_humanoid_guards01.pk4 -r 557858-676149,693160-3947395,4077742-6351893 -Y 1000 -y 10 --connect-timeout 10 -o curldata2.bin -D curlhead2.txt

 

Quote

And am I correct in understanding that this isn't just happening with this one server or is it isolated to darkmod-alt02?

@myfriendstiiv said choosing different mirrors in installer did not help.
I hope he will confirm it with curl commands.
 

Link to comment
Share on other sites

hello I searched on my pc for those .pk4 files appearing in the CURL error,

and it seems I already had the Dark Mod installed on my computer from back in 2016..

I tested it and its fully playable, I can even download maps,

I tried running the tdm_installer from my older existing darkmod directory but received the same CURL error

Link to comment
Share on other sites

8 hours ago, myfriendstiiv said:

@taaaki I had the same problem when trying all four mirrors 

I'm afraid, the problem is fully on your side.

In order to debug it, you can try the following:

  1. If you have another PC in your household, try the failing curl command there. If it works on another PC, then your PC has issues (firewall, antivirus/virus, ...).
  2. If you have a friend with the same internet provider, you can ask him to test the curl command. If it works on his side, then perhaps your router does something wrong.
  3. Finally, you can ask tech support of your provider about the problem. Curl.exe is well-known program, so it should be enough to write the full curl command and the error that you get, it will be trivial to test it on their side.

If you do this, please post your findings here.
I would be greatly interested to know what causes the problem 🙄


Meanwhile, I have added a checkbox "No multipart requests" to tdm_installer.
You can see it under advanced settings on the first page.
It may lead to slower download and a bit more stress on server, but it seems you don't have much choice anyway.

Please retry installing TDM again with this checkbox enabled.
Please attach logfile after it finishes.


Originally I thought about retrying failed downloads implicitly, without new checkbox.
However, judging from curl output, the request takes 5 minutes before failing.
Given that retry mechanism works separately on every pk4 file, it would mean 10 minutes per pk4 file, i.e. an hour of wasted time.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Recent Status Updates

    • Ansome

      Finally got my PC back from the shop after my SSD got corrupted a week ago and damaged my motherboard. Scary stuff, but thank goodness it happened right after two months of FM development instead of wiping all my work before I could release it. New SSD, repaired Motherboard and BIOS, and we're ready to start working on my second FM with some added version control in the cloud just to be safe!
      · 0 replies
    • Petike the Taffer  »  DeTeEff

      I've updated the articles for your FMs and your author category at the wiki. Your newer nickname (DeTeEff) now comes first, and the one in parentheses is your older nickname (Fieldmedic). Just to avoid confusing people who played your FMs years ago and remember your older nickname. I've added a wiki article for your latest FM, Who Watches the Watcher?, as part of my current updating efforts. Unless I overlooked something, you have five different FMs so far.
      · 0 replies
    • Petike the Taffer

      I've finally managed to log in to The Dark Mod Wiki. I'm back in the saddle and before the holidays start in full, I'll be adding a few new FM articles and doing other updates. Written in Stone is already done.
      · 4 replies
    • nbohr1more

      TDM 15th Anniversary Contest is now active! Please declare your participation: https://forums.thedarkmod.com/index.php?/topic/22413-the-dark-mod-15th-anniversary-contest-entry-thread/
       
      · 0 replies
    • JackFarmer

      @TheUnbeholden
      You cannot receive PMs. Could you please be so kind and check your mailbox if it is full (or maybe you switched off the function)?
      · 1 reply
×
×
  • Create New...