stgatilov

I believe you can look in tdm_installer logs or game console (at least on the latest dev build) to see which URL is used. I have just returned from vacation in Turkey, and suffered from abysmally slow internet there. It took me several days to download 10 MB installer for a program, because download speed was 50 KB/s and the download was frequently stopped (I use Firefox). Perhaps hotel managers do it intentionally so that people give up on the virtual world, go to the beach and rest better This forum was very slow to load. Posts were loaded almost one by one. I had to refresh some pages 5+ times (worse for longer threads) because loading process often stopped before being finished. Updating gmail and browsing other sites was not much better actually. On the last days there, I decided to try tdm_installer to see how it behaves in such conditions. I did not attempt clean install because it would obviously take longer than my vacation, but I did some 50-80 MB updates. I could not even imagine internet could be so bad! HTTP requests were dropped frequently (timeout due to effective download speed averaged for the last 10 seconds dropping lower than 1 KB/s). Once timeout happened during downloading tdm_installer.ini (8714 bytes text file). I did some improvements to make installer more stable, but it still managed to stop with error: it failed to download 4 KB piece of a file two times in a row! I have also noticed that mirrors behave differently. Cabalistic's mirror is Germany was the most reliable for me, and the main TDM server in Canada was the worst one. Note that the main TDM server is not used for FM or game downloads. However, it is solely used to obtain some "trusted" files: installer exe, .ini file and 2 MB manifest. It turned out that downloading 2 MB manifest from the main server was the hardest challenge for the installer. I can speculate that Canada is on the farthest end from Turkey in terms of connectivity. Which is interesting because physical distance from Novosibirsk to Canada is about the same as the physical distance from Turkey to Canada, and for me all our mirrors and the main server work perfectly well and fast. Note that in-game download is much less reliable than tdm_installer, because it can only download whole FM in single curl request, it does not try to break it into smaller pieces and retry. Resumable downloads are not supported neither by tdm_installer nor by in-game downloader: that would be a mess to implement and maintain. For me, the conclusion is: we need to support alternative ways of downloading, because no matter what we try, there would be people who cannot use our custom downloaders. For missions it is already available: you can download mission from mirror by direct link on mission database on website. For the game itself, I guess we need to create and seed torrent for the most recent stable version.

Please delete tdm_update and forget about it. I believe it should give you a warning when you run it, saying that it is not going to work anymore. tdm_update cannot be used to install TDM 2.09a. If you ignore its warning, it will install original/old 2.09 version. You need to get tdm_installer.linux64.zip and run tdm_installer.linux64. If you don't have it, download it from website and unpack to the directory where TDM is installed.

You need recent trunk, because 2.09 sources are not very suitable for non-x86 architectures. The latest report about non-x86 CPUs is in this issue. Basically, you need to: Fix the "unknown CPU" error --- you already did that. Build all third-party libs on your machine using conan. Note that TDM does not uses system libs. You can go to ThirdParty/readme.md and read it. There should be explainations how to install conan, and how to build stuff. There is special section for unsupported CPUs (basically, you will need to pass some string keyword both to conan install and to cmake). Unfortunately, if some dependency does not build on your machine, they you have a big problem, since patching conan recipes is possible but not not easy at all. Alternative option would be to hack system packages by modifying ThirdParty/cmake_find_package/*.cmake... I think if you manage to overcome the dependencies issues, then TDM should build fine. No idea what sort of errors you'll meet in runtime though... While your platform is not officially supported, I could surely fix the build on it if I had access to it. I wonder where did you get the workstation from...

I can't analyze such problems in detail right now, but I also wonder how did you install the game. Rev 9108 looks like original 2.09 release (unless I'm mistaken). You need to update to 2.09a (hotfix release), because it includes some fixes for in-game downloader. Just run tdm_installer from the directory where TDM is installed, and go forward on default settings. You probably want to restore your config on the last page (not a good idea in general, but 2.09 -> 2.09a should be fine).

Yes, it makes sense. But make sure to disable it woth in HandleCompression and in SelectInternalFormat. I'd say we can probably refactor SelectInternalFormat into two parts: the one which is O(1) and return format without any analysis, and the one which can additionally check the image data for final internal format. In this case I'd call the first function in HandleCompression instead of writing the same conditions again.

I have committed all my changes to source code repo. New software RGTC compressor is enabled for everyone. I did not provide a way to disable it, but you can easily find it if you want. There is nothing like image_mipmapMode in the code already, maybe @duzenko removed it later.

I implemented RGTC compressor in SIMD (generic and SSE2), as well as mipmapping. The compressor uses min/max value of a block as min/max ramp points with 6 intermediate ramp points between them, and snaps each color value to the closest ramp point (well, almost closest sometimes). On the shitty i3-5005U CPU which I have now, compressing 2K x 2K normal map takes 28 ms (with all mipmaps included). This means 7 ms per 1K x 1K texture, or 7 ns per pixel. Given that loading the 2K x 2K TGA image from PK4 file takes about 200 ms, I think 30 ms is negligible in comparison. The only question is quality. It seems that naive compressor should be good enough for RGTC. I guess I'll wrap up the changes and commit them, so that someone else could test it on hard cases if there are known ones. It took me several days here to download 10 MB installer of DDS Viewer, so I won't be able to test it myself

As far as I remember, it is autoexec.cfg.

If it is not hard, please find where it started. If that's dev16330-9508, then it most likely means that your CPU was not able to keep up with game modelling. Game tic models 16 ms of gameplay time (slightly more on uncapped FPS). If your CPU spends more time for modelling a tic, then game enters so-called "spiral of death", and FPS quickly falls down to zero. The engine has some hard limit however, something like 10 tics per frame, which stops the spiral, slowing down gameplay instead. But if the limit is really 10 ticks, then this stop should be somewhere at less than 6 FPS.

Did it start from dev16330-9508 ?

Some notes about the main algorithm. Pixel values [0..255] are treated as float values [0..1] (standard thing in image processing). The 8 values which can be selected from are called "ramp points". When min/max ramp points are determined, all the other ramp points are computed by formula and rounded to the nearest integer (see GetRmp1, _bFixedRamp is true in our case). Then all input pixel values are encoded as the closest ramp value (see Clstr1). If several candidates are best, then it's hard to say which one is chosen due to numeric errors from dividing by 255. The format supports two orders of min/max ramp values (see here). The auxilliary one always has 0 and 255 values. That's good for alpha in transparency, but seems pointless for normal maps. Still, the compressonator's algorithm tries both ways and chooses the one which is better (see CCodec_DXTC::CompressAlphaBlock). Now, the hardest part is selecting min/max ramp points, and it is far from simple (see CompBlock1). It includes some search when max-min is greater than 48, plus some "refinement" trying to minimize error while ignoring rounding. I won't be able to reproduce this, that's for sure. However, there is also a setting for compression speed, and if you set it to fast or super fast, then different code is used instead of all the stuff described above (see DXTCV11CompressAlphaBlock). For some reason, it is disabled under Windows. There is also good explanation at the beginning of dxtc_v11_compress.c file. This is much less arcane, although there are still several hardcoded candidates for min/max ramp points (it tries to add plus/minus one to them).

I looked in compressonator, it uses box filter (CMP_GenerateMipLevelF in cmp_boxfilter.cpp) : for (int i = 0; i < 4; i++) *pDst++ = (c1[i] + c2[i] + c3[i] + c4[i]) / T(4.f); with rounding down. By the way, given that c1[i], c2[i], c3[i], c3[i] and T are all unsigned char, which this code does not overflow? Note that this is not "blur", this is averaging of 2x2 blocks. This is not even arbitrary resize, where old texels and new texels interact in arbitrary way.

You can go to https://haveibeenpwned.com/ and check your email/account. If credentials leaked from some websites, check if they had the same or very similar password to leaked one.

And... what is meant by "disable new highlight" ? Given that old frob stages will be gone soon (btw, maybe good time to do it already?), what will be left if you disable outline?

Yes, it should not be a problem.

I have alternative proposal. It seems that RGTC compression algorithms are trivial. I'll check up the algorithm in compressonator. If it is really that simple, then I'll implement SIMD-accelerated compressor RGBA8 -> RGTC on CPU. After that we can compress textures in memory during load like we do now, but 1) it will be done same way regardless of driver, 2) it will be fast regardless of driver, 3) it can be moved from main thread to worker threads which we already have for loading TGA, making it parallel to uploading data to OpenGL. It seems such approach allows to avoid the whole "compress everything" story and leave assets as they are. It also allows using uncompressed normal maps (although large missions won't boot in such case). Also, it avoids locking in to RGTC compression format. Precompressed RGTC textures should also work fine if someone decides to use them.

@OrbWeaver has already discussed it in DevIL/png thread.

I added commands for all the features except for "Restart Game" with dmap on. Right now "dmap" checkbox is purely GUI dialog thing, it is not stored in GameConnection singleton. So, I added command "GameConnectionRestartGame", but it never dmaps regardless of what is ticked in the dialog currently. I'm not sure if users need to have a separate shortcut for Restart game + dmap or if they need the current state of dmap checkbox to be respected... Also, I have noticed these "advanced toggles" (used on checkboxes) are not very good: The hotkeys are broken with them. If you only use hotkeys, then it works properly. But if you start mixing clicks on checkbox and using hotkey, then sometimes hotkey simply tries to always set false or to always set true, without changing state. Since advanced toggles are set directly in EventSystem, they cannot be used in DR console. They can be toggled by shortcuts though, so I'm not sure it is a big problem. Also, there is some naming confusion. I originally called two approaches "reload map" and "update map", although the latter term is pretty stupid and I would be happy to find a replacement. But the thing is: both approaches are "hot reload" to me (in fact, the difference between them is mostly on the DR side). For now I keep this naming convention in the code (if it is changed, it has to be changed everywhere at once). For the GUI dialog, I have used @OrbWeaver terms to make two approaches sound distinct. However, the commands still use the naming convention from the code. And if power users decide to configure shortcuts or execute commands, they will face the differences between dialog and code terms.

I wonder what is console used for? Is it worth exposing game connection toggles/buttons for console? What about Python scripting? Can it generate an event? Can it run a command?

What's the difference between command and statement? I mean, I have this code: // Construct toggles _camSyncToggle = GlobalEventManager().addAdvancedToggle( "GameConnectionToggleCameraSync", [this](bool v) { bool oldEnabled = isCameraSyncEnabled(); setCameraSyncEnabled(v); return isCameraSyncEnabled() != oldEnabled; } ); // Add one-shot commands and associated toolbar buttons GlobalCommandSystem().addCommand( "GameConnectionBackSyncCamera", [this](const cmd::ArgumentList&) { backSyncCamera(); } ); _camSyncBackButton = GlobalEventManager().addCommand( "GameConnectionBackSyncCamera", "GameConnectionBackSyncCamera", false ); As you see, _camSyncToggle is created from lambda and directly tied to toolbar button. But _camSyncBackButton is created as a command in CommandSystem, then same-named command is created in GlobalEventManager, which is finally tied to toolbar button. Why can't we delete the last three lines? What do they even do? UPDATE: Both of these two things work in toolbar, and both of them show up in Keyboard Shortcuts and work properly from shortcuts too.

There is one more thing I forgot. In my opinion, it should be possible to assign hotkeys to game connection actions. Defaults keys are not needed, but if mapper finds himself using something a lot, it should be possible to do it quickly from keyboard. Aside from speed, there is also a problem of screen size. @Wellingtoncrab has a laptop with small screen, and even the new GUI window takes too much space on it. He should be able to use common features with GUI window hidden. How is it usually done in DarkRadiant?

That's an interesting option, although a bit complicated. And if we do it for normal maps, better do it for ordinary textures too. Basically, we can distribute tga/png textures, and write dds textures straight on player's machine on-demand. Of course, TDM installation will increase in size... With all the normal maps, I have dds directory of size 2.5 GB. Now consider how the size will increase if we replace dds with tga for ordinary textures (provided that we find most of them in SVN history). I'd say it will become 5-6 GB download and 9 GB after install...

I think I initially resized the window in wxFormBuilder, then got annoying problems with activity indicator, and finally forget that sizes should not be constant. I hope I'll remember where to look for information if I do any GUI in the future. UPDATE: I added SetMinSize just before calling Layout. I tried merging the branch, it worked fine, no conflicts. Nobody except me changed Game Connection code, and my changes outside it consists of a tiny hack in MapExporter. If you update entity spawnarg every frame, then it will be very bad with "Update after every change". This mode schedules update on next frame after every change (delaying by one frame does not increase latency but allows to move many entities at once). Also, "update map" is blocking. If you update spawnarg only when mouse is released, then it should be OK.

The common opinion is exactly opposite: compression artefacts are way more visible in normal maps than in diffuse maps, and mostly due to specular light. It won't be easy... but I guess it is possible to check for image program and dump tga of the end result, then compress tga to dds and replace image program with dds image path. By the way, will we simply delete tga normal maps after everything is done?

I don't understand the rationale. There is X system, and there is Wayland. The programs written for Wayland don't work in X system. The programs written for X system work in Wayland via compatibility layer. Given that we cannot combine both systems in one build, relying on X system sounds like the best solution.