Jump to content


Photo

Tool for analyzing mission links from the Mission List


  • Please log in to reply
118 replies to this topic

#101 gnartsch

gnartsch

    Member

  • Member
  • PipPip
  • 478 posts

Posted 28 April 2015 - 03:50 PM

 

$ curl --head http://swift-mazes.com/fms/antr.pk4    

...
Content-Length: 33057589
Content-Type: application/octet-stream

Weird. I wonder what curl is doing different than the Java code I use.

swiftmazes is the only server which won't report the Content Length for a mission when doing

   URLConnection tmpConnection = new URL(url).openConnection();
   tmpConnection.getContentLength();

That's why that tool needs to ignore swiftmaze with respect to the checking file sizes... unless I figure out what I am missing.

All I get is the following, and digging deeper into possible options and alternatives to modify the behaviour of the URLConnection did not reveal any insights so far either.

Transfer-Encoding=[chunked], Keep-Alive=[timeout=5, max=100], null=[HTTP/1.1 200 OK], Server=[Apache/2.2.29], Connection=[Keep-Alive], Date=[Tue, 28 Apr 2015 20:39:33 GMT], Content-Type=[text/html]}

The official internal name for "Thief's Den" is now 'thiefsden', as should be the name of the pk4.

See http://missions.thed...le_missions.php


Edited by gnartsch, 28 April 2015 - 03:52 PM.


#102 Tels

Tels

    Mod hero

  • Member
  • PipPipPipPipPip
  • 15024 posts

Posted 30 April 2015 - 10:04 AM

Weird. I wonder what curl is doing different than the Java code I use.

swiftmazes is the only server which won't report the Content Length for a mission when doing

   URLConnection tmpConnection = new URL(url).openConnection();
   tmpConnection.getContentLength();

That's why that tool needs to ignore swiftmaze with respect to the checking file sizes... unless I figure out what I am missing.

All I get is the following, and digging deeper into possible options and alternatives to modify the behaviour of the URLConnection did not reveal any insights so far either.

Transfer-Encoding=[chunked], Keep-Alive=[timeout=5, max=100], null=[HTTP/1.1 200 OK], Server=[Apache/2.2.29], Connection=[Keep-Alive], Date=[Tue, 28 Apr 2015 20:39:33 GMT], Content-Type=[text/html]}

The official internal name for "Thief's Den" is now 'thiefsden', as should be the name of the pk4.

See http://missions.thed...le_missions.php

 

Thanx for the info, if taaki or Biker delete the "thiefs_den" file, I'll nuke it, too.

 

As for the size: I'm not familiar with what the Java code does underneath, but may guess is it doesn't query it with HEAD; but a regualar GET request and then aborts? And this seems to confirm it:

https://stackoverflo...e-of-a-web-file

 

Basically, a "GET" will do a "chunked-transfer", where the lenght can't be sent, while a "HEAD" will work properly (and is actually the right way to get only the header info). With a GET, the server is prepared to send you the entire file, wether you want it or not. Sending, then aborting such a request will just lead to unec. work on the server side.

 

There is a bit of code on that page which shows how to use a HEAD request. I would program your tool so it first tries a HEAD request, and if that doesn't work, falls back to the regular method.

 

Hope this helps!


"The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore, all progress depends on the unreasonable man." -- George Bernard Shaw (1856 - 1950)

"Remember: If the game lets you do it, it's not cheating." -- Xarax

#103 Tels

Tels

    Mod hero

  • Member
  • PipPipPipPipPip
  • 15024 posts

Posted 30 April 2015 - 10:12 AM

Stashing the L10N files into some backup-'archive' isn't my primary intention either.

But from what I know far more mission received a translation, than the number of mission got actually enabled for L10N.

And it still seems like quite a few authors don't want their missions to be touched by anyone else.

That's the issue here, I guess.

So, moving the unused ones away is just a means to keep the primary directory clean.

 

Yeah, but only for the case of "Author doesn't want to" && "it wasn't enabled".

 

If the author gave permission, it can't be hard to "enable" the mission and just use the files? We do have more or less automated tools for that. Would be much better than to just delete the work of the translators...

 

If someone needs help with that, let's just do it.


"The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore, all progress depends on the unreasonable man." -- George Bernard Shaw (1856 - 1950)

"Remember: If the game lets you do it, it's not cheating." -- Xarax

#104 Bikerdude

Bikerdude

    Mod hero

  • Member
  • PipPipPipPipPip
  • 18340 posts

Posted 30 April 2015 - 02:50 PM

Thanx for the info, if taaki or Biker delete the "thiefs_den" file, I'll nuke it, too.

Already done.



#105 gnartsch

gnartsch

    Member

  • Member
  • PipPip
  • 478 posts

Posted 30 April 2015 - 03:47 PM

Thanks for the tip, Tels!

Unfortunately the resulting effect is terrible!

Instead of simply providing the data in a 'cleaner' way, the HEAD requests causes the entire PK4 to be downloaded as soon as I try to get the header info.

Seems like there must be some more details to this issue.

Oh, ... and even after the PK4 was fully downloaded, the file-size was still not to be found anywhere in the response or header. A mystery.



#106 Tels

Tels

    Mod hero

  • Member
  • PipPipPipPipPip
  • 15024 posts

Posted 01 May 2015 - 06:02 AM

Already done.

 

Strange:

 


$ ls -la thief*    
   112908 Jan 24  2014 thiefs_den_l10n.pk4
  3459871 Apr 12  2014 thiefsden.pk4
   112908 Jan 24  2014 thiefsden_l10n.pk4

 

Is the "new" version now without l10n? That would be a step back :(


"The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore, all progress depends on the unreasonable man." -- George Bernard Shaw (1856 - 1950)

"Remember: If the game lets you do it, it's not cheating." -- Xarax

#107 Tels

Tels

    Mod hero

  • Member
  • PipPipPipPipPip
  • 15024 posts

Posted 01 May 2015 - 06:03 AM

Thanks for the tip, Tels!

Unfortunately the resulting effect is terrible!

Instead of simply providing the data in a 'cleaner' way, the HEAD requests causes the entire PK4 to be downloaded as soon as I try to get the header info.

Seems like there must be some more details to this issue.

Oh, ... and even after the PK4 was fully downloaded, the file-size was still not to be found anywhere in the response or header. A mystery.

 

Uh, that is exactly backwards!

 

Maybe it is that problem:

 

https://community.or.../thread/2112393


"The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore, all progress depends on the unreasonable man." -- George Bernard Shaw (1856 - 1950)

"Remember: If the game lets you do it, it's not cheating." -- Xarax

#108 gnartsch

gnartsch

    Member

  • Member
  • PipPip
  • 478 posts

Posted 09 June 2015 - 02:25 AM

An issue in the mission database:

 

seems like Requiem received a silent update?

 

The version on taaki's server is slightly bigger and newer than the other two mirrors.

Once again I see that the other two references are pointing to the TDM2testmissions folder.

I guess the updated version got pushed to the regular missions folder instead.

I would recommend to clean up those folders and move the current versions to the regular folder.

I don't think that all that stuiff is still under test.

 

'http://darkmod.taaak...ms/requiem.pk4'
OK
   Size: 102.5 MB (107437077)
   Last update: Sat May 30 09:35:37 CEST 2015

 

'http://www.fidcal.co...ns/requiem.pk4'
OK
   Size: 102.4 MB (107330042)
   Last update: Sat Oct 05 11:43:42 CEST 2013
   Language: English
   Sources: [PHP, XML]

'http://www.southquar...ns/requiem.pk4'
OK
   Size: 102.4 MB (107330042)
   Last update: Sat Oct 05 11:46:52 CEST 2013



#109 Bikerdude

Bikerdude

    Mod hero

  • Member
  • PipPipPipPipPip
  • 18340 posts

Posted 09 June 2015 - 03:56 AM

And regarding where they are getting uploaded to, all mission are going to the regular folder on all mirrors.

 

I beleive the issue is that the FM page is using old links, which I need to change. I will also have a look at all the other FMs and make sure none of them are using old links.



#110 gnartsch

gnartsch

    Member

  • Member
  • PipPip
  • 478 posts

Posted 15 November 2015 - 04:57 AM

A few minor issues in the list of download URLs:

 

Penny Dreadful 1: The grail of regrets

bad URL : protocol reads 'ttp' instead of 'http'

ttp://www.fidcal.com/darkuser/missions/penny1_1.pk4

Patently Dangerous

dead link

http://www.southquarter.com/tdm/fms/patently_dangerous_v2.pk4

Thomas Porter 3: Glenham Tower

dead link

http://www.southquarter.com/tdm/fms/glenham.pk4


#111 Bikerdude

Bikerdude

    Mod hero

  • Member
  • PipPipPipPipPip
  • 18340 posts

Posted 15 November 2015 - 08:40 AM

Fixing now.



#112 skina

skina

    Newbie

  • Member
  • Pip
  • 8 posts

Posted 07 June 2016 - 04:59 PM

http://www.thedarkmo...details/?id=111

ttp://darkmod.taaaki.za.net/fms/volta1_1.pk4


#113 Bikerdude

Bikerdude

    Mod hero

  • Member
  • PipPipPipPipPip
  • 18340 posts

Posted 07 June 2016 - 05:31 PM

Fixing now.



#114 nbohr1more

nbohr1more

    Darkmod PR, Wordsmith

  • Development Role
  • PipPipPipPipPip
  • 7519 posts

Posted 24 August 2016 - 12:24 PM

A new scan attached.

 

Is there a way to remove the swift-maze mirror from the scan?

 

Attached File  analysis.txt   54.49KB   4 downloads


Please visit TDM's IndieDB site and help promote the mod:

http://www.indiedb.c...ds/the-dark-mod

(Yeah, shameless promotion... but traffic is traffic folks...)

#115 Bikerdude

Bikerdude

    Mod hero

  • Member
  • PipPipPipPipPip
  • 18340 posts

Posted 24 August 2016 - 01:28 PM

Fyi, those mirrors are I believe Springheels.



#116 gnartsch

gnartsch

    Member

  • Member
  • PipPip
  • 478 posts

Posted 26 August 2016 - 01:54 PM

A new scan attached.


attachicon.gif analysis.txt

Whoops!

Someone actually using that little tool?

I am baffled and surprised!  ;) 



#117 Springheel

Springheel

    Creative Director (retired)

  • Admin
  • 36144 posts

Posted 26 August 2016 - 02:51 PM

A new scan attached.

 

Is there a way to remove the swift-maze mirror from the scan?

 

attachicon.gifanalysis.txt

 

Swift-mazes belonged to Tels IIRC.



#118 SeriousToni

SeriousToni

    Beginner Contest 2012 Winner

  • Member
  • PipPipPipPip
  • 2194 posts

Posted 26 August 2016 - 06:02 PM

Indeed, that is correct, but he has not been around lately and sadly.


"Einen giftigen Trank aus Kräutern und Wurzeln für die närrischen Städter wollen wir brauen." - Text aus einem verlassenen Heidenlager

#119 Bikerdude

Bikerdude

    Mod hero

  • Member
  • PipPipPipPipPip
  • 18340 posts

Posted 26 August 2016 - 08:41 PM

i will get around to sorting that out at some point.






0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users