[AngelLoader TDM Support (now released!)]

I've opened a github issue https://github.com/FenPhoenix/AngelLoader/issues/117

[AngelLoader TDM Support (now released!)]

Public release v1.7.6 (with Dark Mod support) is out.

Improvements since the final beta 14 are:

• Fixed a few remaining bugs with zip/pk4 support.
• Game Versions window now properly displays TDM version.
• Import window no longer has a vestigial off-screen TDM field (because TDM doesn't need or support importing).
• Web search option is now disabled if an unknown/unsupported FM is selected.
• If an FM with an unknown or unsupported game type is selected, the messages in the tab area now no longer refer to Thief 3 ("Mod management is not supported for Thief: Deadly Shadows").

The full changelog can be viewed at the release link.

The de facto official AngelLoader thread is here: https://www.ttlg.com/forums/showthread.php?t=149706

Bug reports, feature requests etc. are usually posted there. I'll continue following this thread though.

Thanks everyone and enjoy!

[AngelLoader TDM Support (now released!)]

2 minutes ago, snatcher said:

I can't find beta 14, other than the source.

Oops! Uploaded.

[AngelLoader TDM Support (now released!)]

Here's the latest beta. I feel like this is about ready for a public release at this point. If no one has any bug reports or anything, and if I don't find any issues in the next few days, I'll start preparing the first public release.

Beta 14

• Matching game behavior on a bunch of subtler points. None of the below scenarios are likely to occur, but it's best to be able to handle them.
  • TDM auto-converts FM file names to a valid form (lowercase, replacing disallowed chars with underscores, etc.). AngelLoader is now able to match non-converted FM names to their converted counterparts, so it knows for example "BaKerY;job.pk4" is the FM "bakery_job".
  • TDM doesn't care what the pk4 inside an fm's folder is called. So "C:\darkmod\fms\bakery_job\bakery_job.pk4" will load just as well as "C:\darkmod\fms\bakery_job\totally_different_name.pk4". AngelLoader is now aware of this during scans and will not require the in-fm-folder pk4's name to match the FM's.
  • TDM supports compressed FMs with the .zip extension as well as .pk4 when the FM is in the base FMs dir (hasn't been moved yet). AngelLoader is now aware of this, and of the priority order of the two extensions, and will choose the correct file when scanning/loading.
• In Settings -> Paths, the Backup Path area has been polished a bit.
• In Settings -> Appearance -> Show or hide interface elements, the checkboxes are now "Show" instead of "Hide", to be more straightforward.
• When scanning, the progress box will now show a message that it's getting TDM FM data from the server if it's doing so, rather than the more generic "preparing scan" message.

[AngelLoader TDM Support (now released!)]

Beta 13

• More robust TDM file reading attempts: we now try until we can access them or until a 5 second timeout
• If a TDM file is not found at all, we now continue immediately and don't wait the 5 seconds
• Rework auto-refresh system to be simpler and more robust:
  • Refreshes run completely on the UI thread now
  • Refreshes happen immediately or not at all; no more deferred refreshes
  • Refreshes are not allowed when the main window is blocked or disabled (mostly if a progress box is up)
  • If a dialog window is open (Settings, About, etc.) then a "lightweight" refresh (UI update only) is allowed, but a "heavyweight" refresh (FM list reload, possible scan and/or readme cache update) is not.
• Remove installed status from unavailable TDM FMs (those not found on disk)
• Gracefully handle scenarios where some or all watched TDM files or directories may not exist (clean install, partially broken install etc.)
• Ignore empty or invalid FM dirs (dirs with no pk4 files in them; TDM clean installs may add an empty "saintlucia" folder for example)

[AngelLoader TDM Support (now released!)]

Yeah, there's the last played date, but there's no (reliable) way to know if you actually completed the game on that date. What I mean by "no reliable way" is that the following would have to occur for the last completed date to be accurate:

• AngelLoader is running.
• You're playing an FM (say, Iris) on a difficulty you have not completed yet (say, Expert).
• You complete the FM, the game marks it as "completed on Expert" and writes that out to missions.tdminfo.
• AngelLoader detects the change to missions.tdminfo, reads it, and compares to its internal list. It sees that Iris' finished states differ between the internal database and missions.tdminfo: the latter has Expert marked, whereas the internal database only has Normal marked (or nothing, or whatever). It adds the Expert finished state to the internal database for Iris, and updates the UI. Because it catches every modification of this file and updates its internal database every time, it knows the difference in finished states occured on the most recent write. Thus, it could timestamp this and there's your last completed date.

Unreliabilities:

• If AngelLoader were not running when the game writes the value out, then you have no guarantee of time anymore. Not only were you not there to catch the change event, but the file's last-write-time doesn't save you either: the file could have been modified any number of times (updating any number of FMs' entries) since the time of the updating of the actual FM you completed, and individual entries are not marked with the time of their finished state change. So if multiple FMs were finished between AngelLoader runs? Even if you went with the file timestamp, that's still only one. If you finished two FMs between AL runs, any information whatsoever about the time of completion of the first one is completely lost. Same if the game simply wrote over the file with unchanged data, that would lose your correct file timestamp too.
• If you finished an FM on a difficulty you have finished before, then the game will simply write out the same value resulting in no change to the FM entry in missions.tdminfo. AngelLoader will still detect the file modification and will still read and compare the file, but now no differences will be found, so it's impossible for it to know whether any FM's finished state has been changed, and thus it can't know whether you finished it that run or not, and can't datestamp it.
• The last finished dates of any FMs you completed before you set up AngelLoader are unknowable, period. It might be the last played date but it just as well might not be.

We'd end up with a mishmash of correct data and guesses that may just as well be wrong as right. If you can't trust the data then I'm not convinced it's that useful.

[AngelLoader TDM Support (now released!)]

Yeah, a last completed date would be something the game would have to implement (and write it out to missions.tdminfo). Apart from that, there's no reliable way for AngelLoader to know it.

Beta 12

• Fixed: the first "refresh from disk" trigger from TDM would be ignored, due to spurious refreshes being queued on startup and then never run, thus not having the refresh level value reset.
• In the Settings window, if a setting was changed that required a view refresh (game paths etc.), the filters would lose their state after the refresh.
• Dark Mod FM scans now show a progress box with a cancel button, because it's theoretically possible for the server data download to take some unboundedly large amount of time (highly unlikely, but covering the edge cases).
• Title filter searches now also search TDM "archive" names, same as for the other games.
• The DarkLoader Import window will now disable the Import Saves checkbox if no backup path has been specified, and shows a link to go to the Settings window where you can specify a backup path.

[AngelLoader TDM Support (now released!)]

Beta 11

• Fix finished-on state auto-update was unreliable
• Slighty improve scanner title/author detect

Tags are now named some whatever regular-version-looking thing to force GitHub to put the newest at the top

[AngelLoader TDM Support (now released!)]

In-app downloading is turning out to be a lot more troublesome than I thought, and some of the auto-detection of stuff the game does is more troublesome than I thought too. So I'm thinking I could just make the first public release sans downloader and then take my time to develop something nice.

Anyway, new beta.

Beta 10

• Auto-update finished difficulty when an FM is finished in-game.
• If AngelLoader doesn't have a last-played date already, it will pull it from TDM's database. But if it has its own, it will use that, because it's more granular (TDM's only goes down to the day).
• Sorting for the Archive column for TDM FMs now works correctly.
• "Play Without FM" for TDM now deselects any selected FM before running it. Don't know, just seemed cleaner. Think of a like a roundabout "download missions" button where you can go in-game with no FM and download some, I dunno.
• Added support for detecting in-place updated FMs, but only scan on next startup due to race condition with the pk4
• Removed support for auto-refreshing on pk4 add, because the game adds pk4s individually to the fms dir before moving them to their own dirs, and we were trying to scan with each one (and failing due to locks and such)

[AngelLoader TDM Support (now released!)]

Beta 8

Some fixes...

• Selected TDM FM was not updated on refresh from disk.
• Selected TDM FM was not updated if the FM was deselected in-game.
• An optimization was broken which caused a refresh from disk even when unnecessary.
• We no longer pass the FM as an argument to the game, but rather simply write it out to currentfm.txt. This avoids an issue where if you pass an FM as an argument to the game (+set fs_currentfm some_fm), then you deselect the FM or select another one, upon game restart it will re-select the FM you initially passed, thus effectively preventing any change of selection unless you quit and start again manually.

[AngelLoader TDM Support (now released!)]

Anyway, Linux is backburner until I get TDM support release-ready and an AL public release is out. AL on Linux would be awkwardly in between worlds, supporting 4 games that must run on Wine, and one game that's native. That sounds like a headache any way you look at it.

I know some people have been able to run AL on Wine, but from screenshots I've seen it doesn't even look right, some of the GDI drawn stuff looks janky without the high quality scaled filter it's supposed to have, and dark mode doesn't fully work either afaik, etc. So I don't think AL on Wine is that great a solution in the first place, even if it could interface with native TDM.

Now, AL as it stands is stuck on Windows - and WinForms at that - because of the need for the RichTextBox (WinForms has the only acceptably performing one). Making a native Linux version would need a cross-platform UI, so Avalonia or something else as long as it does theming and a virtualizable datagrid I guess. And then I would have to remove rtf support for that version, and then interface with Wine for Thief but native for TDM. Like I said, sounds like a huge headache.

However, a separate Linux app that only supports TDM but otherwise has all of AL's functionality would be MUCH simpler to make. AL's code is already modern-.NET-ready except for the UI, so I would "just" have to learn Avalonia or whatever, and then simply copy AL's code wholesale and rip out anything not TDM-related (which is a large portion of it). That would lessen the "one-stop shop" factor, but would still be way more feature-rich than the in-game mission manager. But that's for the future if I feel like taking it on. For now, I'm just going to get TDM working at a public release level for AngelLoader and offer my sincere apologies to Linux users.

[AngelLoader TDM Support (now released!)]

Quote

Thanks to the above, Winelib supports most C and C++ source code

No mention of anything else, and

Quote

The main difference is that the compiler becomes much more important. It is highly recommended that you use gcc, g++, and the GNU binutils.

Those are C++ compilers. No mention of .NET anywhere on the page either.

[AngelLoader TDM Support (now released!)]

That's only for native C and C++ I guess.

[AngelLoader TDM Support (now released!)]

Beta 7

• TDM FM Scanner now uses a hybrid of local and server data for more accuracy. Titles, authors, and release dates improve.
• AngelLoader will auto-refresh its FMs list when new pk4 FMs are added manually.
• Localization packs (*_l10n.pk4) in the fms directory are now ignored: the game moves these to their appropriate folders on next start.

I reckon this is now pretty usable. I'm working on an in-app downloader, but until that's done, you can use the game to download FMs and AngelLoader to launch them and manage their metadata.

[AngelLoader TDM Support (now released!)]

1 hour ago, stgatilov said:

The description of missions on server is located here:

• http://missions.thedarkmod.com/available_missions.xml (cvar tdm_mission_list_urls).

Every mission has "id" field, you can put it into template to see details:

• http://missions.thedarkmod.com/get_mission_details.php?id=25 (cvar cv_tdm_mission_details_url)

And from this XML you can put screenshot path into yet another URL to download screenshot:

• http://missions.thedarkmod.com/screenshots/25_builders_blocks_0.jpg (cvar tdm_mission_screenshot_url)

These are cvars in engine:

idCVar cv_tdm_mission_list_urls("tdm_mission_list_urls",	"http://missions.thedarkmod.com/get_available_missions.php;http://missions.thedarkmod.com/available_missions.xml", CVAR_GAME, "The URLs to check for the mission list XML." );
idCVar cv_tdm_mission_details_url("tdm_mission_details_url", "http://missions.thedarkmod.com/get_mission_details.php?id=%d", CVAR_GAME, "The URLs to check for the mission details XML." );
idCVar cv_tdm_mission_screenshot_url("tdm_mission_screenshot_url", "http://missions.thedarkmod.com/%s", CVAR_GAME, "The URL template to download the mission screenshots." );

This interface for missions database has not changed in ages.

Actually I'd already figured that part out from the source code and a bit of trial-and-error, but thanks nonetheless!

I do have one question. The tdm_mission_list_urls cvar has two urls:

http://missions.thedarkmod.com/get_available_missions.php
http://missions.thedarkmod.com/available_missions.xml

They both appear to lead to the exact same xml file, except the first one has an xml header and the second one doesn't. From looking at the code it seems like it tries them in order. Should I be doing this too? Are they just mirrors and one might be down? Since they're both on the same domain it seems odd they'd be mirrors, surely if one was down the other would be too, but I dunno.

[AngelLoader TDM Support (now released!)]

Added per-game web search urls and defaulted TDM to site:thedarkmod.com.

I'm working on downloading/parsing FM information and seeing what I can do with it. I really think an in-app downloader would make things a lot more convenient.

[AngelLoader TDM Support (now released!)]

6 hours ago, snatcher said:

Many thanks for supporting The Dark Mod, @FenPhoenix !

Web searches narrow results by site ttlg.com, you could make an exception for TDM FMs...

EDIT - Ouch, my bad: it can be configured in the settings. Regardless, perhaps this is the time to have per-game search results settings with some defaults.

I noticed that too. I'm definitely wanting to add a second URL field at a minimum, or per-game as you say. It'd be cool to be able to just go straight to the forum thread too, which I could do if I got it from that wiki page. I'm just nervous about getting stuff programmatically from the edit section of a wiki page, that just seems like it could change and the url is like some &action=edit which isn't an explicit filename (it's a command to generate a page maybe?) so maybe that could change too? If the text in the source code box were accessible by itself in some explicit file I guess I'd feel less apprehensive. Then again I'm totally inexperienced when it comes to web stuff.

For example, http://missions.thedarkmod.com/available_missions.xml is where TDM itself gets the available FMs list from, though that list doesn't have any forum links or much extra info. Although incidentally I could use it to populate the FM titles, authors and dates instead of a local scan, maybe get a bit more accuracy.

[AngelLoader TDM Support (now released!)]

But I mean you want me to take a dependency on parsing the edit page of a wiki article though? And trusting it to stay there, stay updated, and the format not change? Heck, the page even changes depending on if you have edit permission or not, and I have no way to know about updates either without just constantly re-downloading it.

[AngelLoader TDM Support (now released!)]

22 minutes ago, Araneidae said:

Looks nice.  Is there any prospect of running it on Linux?

You can try with Wine (some have had success, though there may be visual issues or other imperfections). Oops, I guess you can't use Wine because TDM on Linux will be an actual native version, which AL presumably would not be able to interact with. Other than that, sorry, I'm pretty tied to Windows for AL as a whole. A separate app for TDM only would be much easier to port to Linux, no need for the Windows-only RichTextBox and no need to interact with Wine from the Linux side programmatically either. One could use the Avalonia UI framework for that, I would think. Maybe I'll give it a shot sometime, but for now I'll polish up TDM support on current AL.

[AngelLoader TDM Support (now released!)]

I'm the developer of AngelLoader, a fan mission loader/manager for Thief 1/2/3/SS2. I recently decided to try adding Dark Mod support.

The latest release is here:

https://github.com/FenPhoenix/AngelLoader/releases

I'm going to keep a log of issues, ideas, notes, etc. here and we can discuss them or whatnot. If anyone has ideas or suggestions, feel free to post them here.

TDM version requirement:

I've only tested it with TDM 2.11. I'm not sure but I dimly remember FMs used to be stored differently at some point in the past? If so, it wouldn't work with those older versions. I assume people would normally keep their TDM install up to date, but just a heads up.

How it works:

In the Settings window -> Paths tab, choose your Dark Mod executable. Click OK. Now your TDM FMs will be detected, scanned, and added to the list.

For the other supported games, there is the concept of "installed" vs "not installed", this is because they normally come in archive files and then the loader can "install" them (extract the file into the "installed FMs" folder). For TDM there is no such concept (FMs are always installed), so the green checkmark "installed" indicator means that that FM is the currently selected one (the one that is loaded up when you start TDM), and only one TDM FM can be "selected" at a time. This indicator is kept in sync with the game, so if you change your selected FM inside TDM, AngelLoader will update its "selected" indicator in-app.

Notes & Issues:

Deleting FMs:
Deleting TDM FMs is not supported yet. I notice TDM itself doesn't allow you to delete FMs either; it has been noted that people have accidentally lost their data and so the option was removed. AngelLoader puts files into the recycle bin when it deletes them, and I could add the option, but figured it was okay to leave it out initially since the game doesn't let you do it either.

Note, though, that you can still delete TDM FMs from AngelLoader's database if you've manually deleted the actual FM from disk first. To do this, first click the "Show only unavailable FMs" button (red X page icon on the top bar).

Last Played and Finished-On-Difficulty:

Finished-on difficulty is now autodetected. Last played date is taken from the game's database only if AngelLoader doesn't have one in its own (its own is more granular).

In the FMs folder, there is a file missions.tdminfo which stores certain pieces of data about FMs. Included among this data is which difficulty the mission has been finished on, and the last played date (down to the day only). AngelLoader also stores this information, but it doesn't get these values from this file at the moment. Instead, the last played date (down to the instant, not just the day) is set whenever an FM is started, and the finished-on state must be set manually.

I could have it watch the file for changes and update the finished-on-difficulty automatically for the current FM. This would override the user setting, but in theory should be accurate anyway. I could also auto-update the last played date, but because it's only down to the day, that would be less optimal than just setting it on FM start, which would give a more granular date and would have sorting working better. So if you played two FMs in one day, the latest played one would sort at the top.

Mods tab is disabled:

For NewDark games, there is built-in support for mods, and they can be enabled or disabled per-FM in an official capacity. I've disabled the Mods tab for TDM because - from a cursory look anyway - I don't think there's such a per-FM method of mod management.

Language selection is disabled:

In the Edit FM tab, there's an option to choose which language to play an FM with. I've disabled it for TDM because there's a language option in-game (not the case with NewDark) and I haven't looked into how I would scan a TDM FM for supported languages anyway.

"Play Without FM" Option:

For the other games, there is the option (in the bottom bar) to play them with no FM. For TDM there is no such thing as "no FM", but the option is still there currently. It will simply start TDM without passing it anything, and whatever FM it has selected will be there. I might remove the option since it doesn't really make sense. This might make people wonder "where's the button for TDM" but on the other hand having it isn't really useful. Not sure.

Mission downloading and updating:

An in-app mission downloader is in progress.

There is no mission downloader or anything like that currently. It simply lists what you've got on disk. This means that to get new FMs or check for updates to existing ones, you still have to go in-game. I don't know how people feel about the convenience factor of this, having to go in-game to get your FMs and then back out to AngelLoader. However, duplicating the TDM download functionality would mean it might well break if TDM ever changes anything about its downloader.

Custom resource stats:

In the Statistics tab, custom resource display is disabled for TDM. I haven't looked into what sort of "custom" things a TDM FM can have and how I can detect them. I may do this in the future.

 

[Notes and things.]

I wanted something to work on, so I decided to just play around with giving AngelLoader integration a shot.

It's not got every feature you might want, but for a day and a half's work it's not too shabby.

https://github.com/FenPhoenix/AngelLoader/releases (and check the latest tdm_beta_* release)

The "installed" state works a little differently from the other games. FMs are always "installed", so the green check mark means that the FM is the current one, or "selected" in the game's terminology. Only one TDM FM can be "selected" at once, just like in the game. There is no downloader or new FM version indication or anything like that, you still have to go in-game for that. But you do get easy readme viewing, rating, comment writing, accurate autodetection of title/author/release date/mission count and nice sorting and searching etc. Basically most of the AngelLoader stuff is there and works.

[Notes and things.]

2 minutes ago, datiswous said:

Why is that format chosen? For easy editing purposes?

FMs have used it since time immemorial; I have to support it. It's used when authors want to make a fancy readme, ie. with colors, bold/italic, images, centered text, links, etc.

[Notes and things.]

Hello all. Author of AngelLoader here.

There were never any plans to support TDM in AngelLoader, as I figured the in-game system was adequate enough. Adding TDM support would not be impossible, of course, but would require quite a lot of work to shoehorn it into a loader that's mostly designed around NewDark games and the way they work and are structured. For example, TDM doesn't really have the concept of some external "FM archive folder", it just keeps the FMs in an "fms" subdir, doesn't it? And it doesn't really have the concept of a set of "local installed FMs" vs a set of "local not-installed FMs", but rather just "existent FMs and then one that's 'the current one'". Or something similar anyway. I'm sure there's lots more differences too.

Also, if you wanted to download new FMs, you'd have to go in-game to do it anyway, then go back out to your external loader, which seems kinda awkward. I mean unless the external loader also duplicated the download functionality so it could replace the in-game one completely... but honestly that's more than I'm willing to take on.

Also, regarding Linux, AngelLoader pretty much can't be made to run on it natively for reasons, unless you're one of the lucky ones who can manage to get it working on Wine (some can, some can't - I haven't been able to). It's tied to WinForms because of the RichTextBox (it needs to be able to show .rtf files) and .NET kinda has no UI framework for Linux, not even MAUI supports it... there's Mono but its WinForms isn't a perfect clone either... I can't move to modern .NET post-Framework versions either because the dark theming breaks. WPF has a RichTextBox but it's unusably slow. etc. It's a whole mess.

So, really, given AngelLoader has to support rtf and a bunch of other legacy Windows-y stuff for its Windows-y games, and it's designed with Thief-y assumptions, it would probably be better to write a new loader from scratch that only does TDM and can run cross-platform. You'd get rid of all the irrelevant complexity of a Thief series loader (including the need to display rtf!) and you could use Qt or something to have a nice uniform cross-platform look and all. But I'm not a C++ coder, I only know C# and just barely enough C++ to fumble out some simple things, so I'm not the one to do it.

Anyway, AngelLoader is open-source MIT so anyone can feel free to fork, learn from, or take whatever they find useful from it.

[Fan Mission: Shadows of Northdale ACT II - Down The Rabbit Hole by Goldwell (2019/03/20)]

Boy howdy, I remember recording for this! Feels like back in the mists of time...

 

Looks fantastic Goldwell. That intro is production value supreme right there!

[Fan Mission: Down by the Riverside, by Dragofer (2016/09/25)]

I didn't get the door conversation in my LP at first either. The one guy walked up to the guy at the door and said his first line, then got no response and walked off again. I believe it was because the guy at the door was on first alert (or whatever TDM calls it), as he settled down shortly afterwards. I replayed that part in my next video, being careful not to cause any alerts, and the convo worked then. It also worked the first time I played off-camera with an earlier version.