Jump to content
The Dark Mod Forums


Active Developer
  • Content Count

  • Joined

  • Last visited

  • Days Won


Everything posted by stgatilov

  1. I think it should be wrapped into a critical section it you want to open files form different threads...
  2. Something like 30 attempts should be enough for sure (look at the console files generated --- they are numbered), usually it crashes on 1-5 attempt. You might also want to use Grayman's config file to have similar conditions. BTW, different VS may be important too.
  3. Run pip install psutil. And do the same for all other missing modules. Also, look into readme.md, it has this info ?
  4. Opening files is not threadsafe. In BFG it is the same. Basically, there is a set of functions which return a string as "const char*", pointing to internal buffer. I wonder what should we do with them to make them threadsafe but avoid allocation. Maybe make internal buffer threadlocal? It should not be too costly. UPDATE: 20 matches of static char ???[???] across the code. I think changing signature is not a good option, since it would require fixing a lot of usages...
  5. Try it with the script, it will repeat automatically until it crashes. Before that, edit registry so that you get a "Application has stopped working" dialog. Then you can click "debug" on the dialog of attached to it directly from devenv.
  6. I hope you will fix it in some nearest future. Yes, idStr better be thread-safe. I'll look into it, of course. As for idDict and idStrPool... I'm not certain. Looking at the D3BFG code, they did not make custom allocators thread-safe. I see no atomic operations and mutexes in Heap.h/cpp, and non of that in idStrPool and idDict. Making all that stuff thread-safe will make it significantly slower. I guess they simply fixed stupid non-thread-safety like static and global variables, and ensured that access to custom allocators and string pool is single-threaded (e.g. from game code only). Did you try to run the repeat.py script? I ran it directly on SVN with Win64 Debug build. I cannot repeat the debug heap dialog message, since most of the time TDM process simply "disappears" somewhere during loading. I assume it is a crash, but I have disabled something like WER or JIT in my system, so I don't see the message box. I guess I should reenable it somehow.
  7. I think the main question to Duzenko is: why is background thread doing anything at all? I recall you said that the background thread is used only when cvar is enabled. I don't see suspicious cvar values in the config file. UPDATE: I think first of all background thread should be fixed: it must be trivial to validate that it does absolutely nothing or is not spawned at all.
  8. Ok, I have reproduced the problem. Although I see nothing suspicious in config. I'll commit automation script to assets SVN shortly. You can run it is python repeat.py enter_game_exit.seq from the directory where you see the .seq-file. This is heap corruption due to multithreading. The newly introduced "background image loading" thread loads a TGA file of menu background. Somewhere during loading, it gets a path "C:\TheDarkMod\darkmod\fms\business3\guis\assets\loading_screen.tga". In the main thread, this string somehow gets into a local idStr of idFileSystemLocal::OpenFileReadFlags. When this string is destroyed, it notices that someone wrote past the end of buffer. So we have to review the whole idStr for working properly with multithreading.
  9. That's not much. But stack trace confirms the idStrPool theory: it crashes on access to a string inside idDict. A crashdump will help, because it embeds stacks of all threads, and the state of heap, where it is sometimes possible to notice something. Config file is the most wanted thing here. I won't be surprised that with right settings, we will reproduce the problem on our machines.
  10. One of the things which came to my mind is string pools. All idDict-s store keys and values inside a string pool. If you access same string pool from many threads, that is a problem. The correct approach would be to have several pools indexed by thread ID or TLS index. Another way is wrapping all string pool functions into a critical section, but I think it is inferior. Anyway, unless I can see the crash, I cannot say anything for sure.
  11. Well, it enables various defines. You can see the defines in BuildDefines.h. There also a bit of writeup here, but it is more focused on TypeInfo, which is not necessary. You can temporarily uncomment ID_DEBUG_MEMORY and probably ID_REDIRECT_NEWDELETE in Debug build, that should be enough to enable whatever debug heap which is provided by ID code. And it won't require typeinfo as the "with memory log" requires right now. However, note that TDM was developed by many people over 15 years, and nobody cared about such tools. So don't be surprised to get all sort of problems with them. I guess people often directly use malloc/free and operator new/operator delete for allocating memory. The define ID_REDIRECT_NEWDELETE is a very hacky attempt to fix that, but it probably does not influence STL usage, etc. Some universal tool may be more precise.
  12. Grayman, please attach your darkmod.cfg. I will try to reproduce. As for tools, I don't know any silver bullet. Linux had valgrind. I have also used Heapy on Windows, it's easy to customize. UPDATE: Also, a crashdump would be helpful.
  13. I did not say that distributed functionality is what makes git hard to use. I said that git in particular is such that it is hard to use. Not because it had to be so, but because it so happened. You seem to bring another misconception here which I often hear: "You are using svn up and svn commit with SVN, so in git it would be the same but also there would be git pull and git push, nothing hard at all. And you don't need other commands or much knowledge for daily use". The reality is that after a week or month of daily use, people will have a choice of either learning the substantial part of how git works (which is much more complicated than SVN) along with some advanced commands, or suffering from their limited understanding of git.
  14. Holy war mode on ? You probably know little software, or only a very limited class of software. See Myth #5. Keeping interconnected applications in single monolithic repo has a lot of benefits over having different repositories. The main benefit is that every revision is set in stone and perfectly reproducible afterwards. It is also possible to do atomic changes to several projects. Just read about why Google does that. SVN allows you to checkout only a few subdirectories of the repo, so monolithic repo does not forbid people to download only what they need. Git sort-of has sparse checkouts, but there are complicated, most likely limited, and etc. Submodules... I'd better never know of their existence ? Yes, SVN is known for having more conflicts and having worse tools for resolving them. But with trunk-based development, they rarely happen. On the other hand, conflicts are the core nature of all VCS we use. Don't tell tales that conflicts won't happen at all with git. The same thing happens with git when people work via merge requests. Which is how it is usually done, especially for outer contributors. Changing VCS won't help you much with not breaking trunk. Unless you are going to work in your private branch for months without merging back, in which case nobody will ever review&accept your merge request due its gigantic size Yes, and this is very dangerous, because most of developers are poor testers. They will gather bugs in their branches, and then when they consider them "truly stable", merge all these bugs into trunk/develop. And nobody will notice it soon, since everyone is deep in his own branches, branched from develop a month ago. I prefer following the most widely used practice of game development, when everyone is sitting on trunk, so that all bugs get noticed and fixed quickly. Do you know that in Naughty Dogs artists even have an auto-update script for assets repo? So that when someone commits something, everybody forcibly gets the updated version. This is a relevant topic due to recent server outages. It is planned that our admin will provide readonly access to current backups to several members, to be sure that the whole TDM knowledge base does not suddenly disappear. Speaking of normal usage, SVN has zero chance of losing data, since there is no history rewrite. Git, on the other hand, has lots of unsafe ways to edit history, and it is often encouraged. Which provides a good opportunity for users to lose their local history. And in worst case, I guess even remote history can be lost. Recall that git is not a Version Control System, it is a "Stupid Content Tracker". Stupid enough to lose the content it tracks. Holy war off? I think the main reasons why SVN stays on TDM are: Aside from the engine source code repo, we have a few more repos. They contain assets, and migrating them to git would be a disaster. And since they stay on SVN, source code should better stay on SVN too. Avoiding one more VCS is enough reason in itself. Git is inherently harder to use than SVN. Moreover, it is harder not because it has to be so, but because it was ill-designed. I don't like the idea of forcing complexity onto everyone (including myself BTW). Whoever wants to use git can take the additional complexity onto himself and use git-svn, or any other approach to synchronize SVN with git. Also, we have artists: while usually they don't work with source code repo, sometimes they decide to build the engine themselves. There are ways to use git locally with SVN. I do this sort of thing myself with hg. Some time ago I suggested creating an official GitHub mirror for source code repo, with some semi-automatic workflow for accepting pull requests. As far as I remember, nobody was interested. And migration is unlikely to happen.
  15. These are two well-known but completely different ways to implement shadows in computes graphics. Stencil shadows were used in Doom 3 from the very beginning, and TDM used it exclusively until 2.06. Shadow maps were added later: they are used in the majority of modern games. Performance implications are different with these options. I guess shadow maps are usually faster (although on my shitty AMD GPU stencil shadows were faster). If you use soft shadows, then shadow maps should look a bit better. To be honest, I don't know myself which options is better ? I just switch back and forth from time to time to see the difference. Since I have a good GPU and like soft shadows, I prefer shadow maps.
  16. Yes, setting Render Scale in game menu is the intended way now to improve performance by sacrificing precision. Unfortunately, there is a set of bugs in 2.07 with non-default render scale: 5055, 5068 So probably manually changing desktop resolution or playing with GUI scaling is the best workaround for now ?
  17. Hmm... I guess I have seen the second error recently. Don't remember how it ended.
  18. I committed the simple fix in svn rev 15746.
  19. It is not exactly true in 2.08. We start at the size large enough to contain all static geometry, which is quite big usually. There are at least two major issues with the new vertex cache that I know of : 5066 and 5065 If non-monotonous growth is an issue, it would be third in this list
  20. Well, TDM installer does not 'want' special installation path. It does not suggest any default. It installs wherever user puts it. By default user cannot put it into program files, and wherever he can put it, it should install and work normally. If user knows where is his user profile, he can install TDM there. The instructions on website advise to install to C:\games\TDM or something like this, if I'm not mistaken. I cannot agree with you here. These days most of Windows games install into D:\Steam\steamapps\common\... Why does Steam ignore Windows conventions you are talking about? ? I guess exactly for the same reason that we are discussing here: it is too risky to mess with Program Files, admin-protected zones, and anything else which is specific to OS. Plus the structure of special Windows directories changes every 5 years or so, with more and more junctions being added, while D:\Steam worked in DOS, works now, and will continue to work until the last of Microsoft OSes dies.
  21. Yes, this is what I mean by switching accounts, I guess my terminology is wrong here. You cannot live without doing so, because you won't be able to install most of programs without admin rights. But if something can run without admin rights, better run it like that. The main concern I'm talking about is that when a program runs under admin, all the files it writes are by default owned by admin and cannot be accessed later without admin rights. So imagine that you have installed TDM without admin rights into C:\TDM. Then you accidentally run it with admin rights and quicksave on The New Job FM. The next day you run TDM without admin rights and continue playing this FM, but suddenly quicksaving doesn't work for you. Even worse, your game settings don't persist between launches. And that is because TDM cannot overwrite its files, because they are owned by admin, and you run the game under user. So you have two choices: always play TDM under admin, or edit permissions to make everything accessible to user (and in most cases you don't know exactly where is the problem, so you have to modify the whole directory).
  22. I'm pretty sure it was monitor gamma settings (through WinAPI). The sliders in the main menu still work this way on SVN. Which is why I suggested expanding postprocessing in such a way that it would look exactly as OS tables do, so that we could finally drop them. I don't like the current postprocessing, since it is a huge black box with too many variables affecting the output. It also has a controversial bloom which many people dislike, and it does so many render passes that I won't be surprised if it really affects performance (it is great if these passes are disabled when bloom is off). But we should not drop it simply because we did not implement it... hence the idea of two presets: simple one (only gamma + brightness) and "HDR-lite" one with all the whistles.
  23. This is a bug. They should work as usual. Perhaps you should write in Tech Support subforums. Bloom is considered one of the HDR effects. I regularly say that having HDR necessarily means having increased precision of the framebuffer (like 16 bits, of 12 bits instead of 8 bits). TDM does not have HDR. That's why any implementation of bloom effect is a tricky balance between blurring light sources and bleaching bright parts of the screen. There is no way around it. Luckily, you can turn bloom off separately from the rest of postprocessing (?)
  24. From my experience, permissions in Windows are a pain for an ordinary one-guy-one-pc case. The workflow I find plausible is to never switch accounts and never run programs under admin (unless it is some type of system maintenance program). Because if you start switching, you will sooner or later have to edit directory permissions ?
  25. This error is reported when TDM cannot open specified file for writing. Which usually means that it doesn't have rights/permissions to write to the location. So I second Freyk's suggestion about permissions.
  • Create New...