Jump to content
The Dark Mod Forums


Active Developer
  • Content Count

  • Joined

  • Last visited

  • Days Won


Posts posted by stgatilov

  1. Attached patch for moving gamma/brightness changes to shader. I'm afraid I cannot apply it to all cases (case r_postprocess 2 is not working now as it seems). I have hardly managed to get it working without postprocessing...

    One side effect is that main menu is not affected by gamma/brightness changes. Not sure if it is good or bad.

    P.S. As for dithering, it can help only if you apply it when you draw the thing which shows color banding, e.g. when you draw sky and clouds. In the final pass, it is useless.


  2. 1 hour ago, duzenko said:

    Suppose that true. How come blitting from sRGB FBO to default darkens the picture?

    Well, I guess when I said "default FBO is gamma space", I was meaning that you must put gamma-space values there, since they are treated as such by the monitor. I did not mean that OpenGL automatically ensures that it gets gamma-space data, although I think there should be way to achieve it.

    I think the main explanation is that OpenGL thinks default FBO is RGB since you did not tell it otherwise and OpenGL does not do any conversions without you telling it to do them. The FBO which you are blitting from contains gamma-space values, and you told OpenGL it is SRGB (by setting image format and enabling GL_FRAMEBUFFER_SRGB), so it auto-converts its data from gamma-space to linear-space during blitting. But you did not tell OpenGL that default FBO is SRGB, so it assumes it is linear and it does not do any conversion.

    Looking at https://stackoverflow.com/a/46117749/556899, you have to pass some flags to WGL to ensure that default FBO is SRGB-capable, then enable GL_FRAMEBUFFER_SRGB. Did you do both of it?


    How come rendering in linear FBO and blitting to default produces the same result as rendering in default FBO?

    Same as above. You did not tell OpenGL that default FBO is gamma-space, so it assumes it is linear (while it clearly is not) and does not do any conversion.


  3. 12 minutes ago, duzenko said:

    No objections from me since I use neither. I do use ambient min brightness, wouldn't want that lost.

    The cvars will works as usual.

    One thing I also want to try is to optionally add a clever dithering to make color banding unnoticeable. Not a good thing for TDM with its visual style, but perhaps some players would prefer it to obvious color banding.



    The default framebuffer (the one we blit to) is always linear. Unless I'm completely mistaken, it means there must be a conversion and there's no way we can send sRGB frame to monitor without that.

    I'm afraid there is some sort of misunderstanding here.

    The "linear space" is where we can sum colors with mathematical addition and that would work like how light intensities add in the real world. It is also called RGB, but this is even more confusing, because when people hear RGB, they think it is what was always around, which is exactly opposite. The "gamma space" is also called SRGB, this is the representation which monitors expect from rendering output (omitting some minor details like different gamma value in different monitors). SRGB is what has been all around us since the beginning. Almost all image files are stored in SRGB space, they look correct when you open them in ordinary image editor since it just sends this data to monitor. Textures of a modern gamma-correct game may also be represented in linear space: in this case opening them in something stupid like paint would show them as too dark. Although they are most likely in such formats that point won't open them.

    The old approach of rendering is: everything is in 8-bit SRGB (gamma space). No conversion is done when opening ordinary image files, no conversion necessary after rendering is done. However, when renderer computes lighting, it adds intensities with simple addition straight in the gamma space (SRGB). It is incorrect, so lights blending and quadratic falloff look wrong. It would be correct if rendering was in linear space, but it is not. This is how Doom 3 is working, and this is how TDM normally works.

    The first solution to the problem is rendering everything in linear space (RGB). Then you store all textures in linear space (i.e. they look wrong when you open them in paint), you store all FBOs in linear space. At the very end of rendering, you do gamma correction as the final pass, thus converting color from linear space to gamma space in order to send it to monitor. To avoid color banding from final gamma correction, you use HDR, i.e. high-precision color formats.

    The second solution is to use SRGB support in OpenGL. You mark incoming textures as SRGB, and mark your FBO as SRGB. It does not magically changes anything, because in TDM they are all SRGB regardless of whether you mark them or not. But now blending and texture filtering have convert from gamma space to linear space before operation, and from linear space back to gamma space after operation. Thus, intensities are adding up properly. Moveover, when you fetch data from texture marked as SRGB in your shader, OpenGL automatically converts it from gamma space to linear space before returning it. If you write output color to SRGB buffer (and have enabled something on that FBO), then OpenGL automatically converts it from linear space into gamma space. So whatever you have in your shader is in linear space (and most likely in 32-bit float), but all data out of shader is in gamma space (and most likely 8-bit since if you use more precision than the first solution is better).

    So you are wrong saying that default framebuffer is linear. It always contains SRGB (gamma space) data. And our textures always contain SRGB data. The only problem is that we incorrectly deal with this SRGB data all over the renderer. And using the SRGB extension in OpenGL allows to fix that, forcing OpenGL to auto-convert to linear space and back around every operation which could add colors.

  4. Ok, how about this.

    We add a trivial shader, which only applies r_gamma and r_brightness to the scene. If postprocessing is off, we just run it over the whole screen. If it is on, we run it after postprocessing is over. We run it as the very final step in any case!

    So we can finally remove stupid monitor tables and Linux equivalents. Color banding still here, of course. Ambient tweaks and SRGB stay as experimental stuff.


    Speaking about color banding with SRGB. There should be no color banding if you display srgb buffer "as is". Such buffer is already in gamma space, native for the monitor. The difference from RGB is that shader automatically converts inputs from gamma space to linear space and outputs from linear back to gamma. But if you want to do any additional gamma adjustments, then you are screwed without high precision buffers.

  5. 6 hours ago, Dragofer said:

    @peter_spy I'd think so too. I've sent a message to Invision support briefly describing what I and others are (not) seeing and put it in relation to the 2019 forum upgrade and our account creation dates.

    I think it was a better idea to make @taaaki know this, maybe this is some misconfiguration on his part. Without his assistance, communication with support would be problematic anyway.

  6. 15 hours ago, grayman said:

    The only thing I can think of is that my failed tests included a new dmapping of the moonlight version, and my instructions only included adding the map file but not the proc, cm, or aas files. I can think of no reason why this would matter, but I'm going to have to revisit this. Back later.

    It depends on .proc file. If you use the old one, there is no "prelight" in the .proc file. As the result, it works. If you dmap it, then prelight version gets added to the .proc file. The engine sees that and chooses a different code path in one place, which results in incomplete lighting. I'll look into it.

  7. 1 hour ago, duzenko said:

    Good catch on that epsilon bug. Sorry I mis-ported it from SSE.

    Bugs... they happen from time to time 😥 I recall how I reviewed this code and missed it. Perhaps should add a random test on this function (i.e. compare to results of generic implementation).

    Anyway, I have recalled where I saw a big outdoors map with huge light, I have found it in my installation. It was Noble Cave by AluminumHaste. Unfortunately, it has not portals outdoors, maybe because adding them broke shadows. Calling @AluminumHaste...

  8. Dragofer reported a problem with shadows, but it turned out to be an entirely different problem (5106).

    Basically, TDM 2.07 introduced a horrible bug to stencil shadows. Most likely, it only affects precomputed stencil shadows for projected/directional lights. Also, it does not affect mappers having too old CPUs. If you dmapped anything with TDM 2.07 with projected/directional lights, it might be worth to re-dmap it with TDM 2.08 when it comes out.


    P.S. Still, I would be happy to see some testmap with (global) parallel light broken 😀

    • Like 2

  9. Well, I did not achieve anything good about this problem. Only fixed popping on perfectly flat walls with rough bumpmap (by hacking shading equation).

    Looking here and here, I see the following solutions:

    1. Offset the front cap of the shadow volume along outer normal by some constant distance. "Front cap" is the part which matches the backfaces. Then enable lighting for backfaces (probably can be done even now with cvar). So, it should make things better, but may introduce light leaks on nearby objects.
    2. Render front faces as normal, but then additionally render backfaces where stencil value >= 2. In simple cases there is only one layer of shadow polygons in front of it, and such places will be lit by ordinary shading, which should be smooth and dark anyway.
    3. Hack shading equation, so that it makes triangle darker when angle between triangle normal and light direction is small. This is the easiest to do, but my attempts to do so made polygonization or apples even more obvious.

    I did not try points 1 and 2. Maybe something to consider in future. Point 2 sounds interesting.

    • Like 1

  10. I have bumped into an old issue about glitches with directional lights: 3818. At the same time, I have recently fixed a bug in interaction culling, which happens when light source illuminates many portals areas at once. If we are lucky, that bug could be the only one 😁

    It would be great to check it on some test map. But creating a testmap from scratch would be a horrible waste of time. I wonder if someone had experimented with parallel lights and vast outdoors areas and saw shadows randomly disappearing. If yes could you please share your test map?

  11. 1 hour ago, duzenko said:

    if bots have vulnerable circuitry in outside parts they should be knocked out pretty easy

    Maybe it makes sense to define damage zones then. And significantly reduce damage on non-vulnerable bones.


    swords should be pretty useless against thick metal structures

    I agree, but I don't see any indication that creators of steambots wanted them to be invulnerable to ordinary weapons.


    do bots make any use of electricity? I'd think electric contacts are susceptible to short-term disruption when kicked

    I think not. They are "steam bots" ;), so they somehow work purely on steam.

  12. I second Goldwell's proposal: run the vcredist executables located in your TDM root directory.

    Aside from that, TDM 2.08 will no longer depend on VC redistributable (just like it was until 2.05), so you should be able to run SVN version on a fresh Windows.

    UPDATE: Finally, if nothing helps, you can use procmon to see where it looks for these DLLs. But this would take considerable time and effort.

    P.S. Upgrades "through several major versions" are rarely pain-free. Windows 10 already had 5-10 revisions released.

    • Like 2

  13. Yes, of course I see it.

    There is another problem, which bugs TDM since 2.06 maybe. When user saves/loads a game, the screen blinks showing some random things. There is also a long-standing issue 4967 on our roadmap about it, although it probably describes some different effects (but for the same reason).

    I agree that it is sad. While it does not harm anyone and has low priority, it makes the apparent difference between a polished game and WIP build.

  14. I'm not sure what you mean by "CMS". If this is CMS aka wordpress, then there is website and forums. If you mean wiki, then here is the wiki. If you mean bugtracker, then here is one. Switching from one system to another is always a lossy process. Also, pretty much everything I listed above on TDM side is better and more well-known than builtin solutions which GitHub provides. MediaWiki is the widest-known wiki engine. Mantis is very popular too, although today's trend is using the awful Jira-monster. Forum is probably not so good (I have never heard about our Forum engine until now). As for website, is there anything except for GitHub pages on GitHub?

    This is perhaps not important for all the hipster developers hanging around, talking about trends and coolness... but I regularly browse through years-old history. I do SVN blame and read commits from 2008 explaining why something was done. I dig issues from TDM 1.04 times. I search over the forums to find what some stale shader was doing originally and how to use it. Migration from svn to git can save most of the history, but migrating other things sounds like an even more stupid idea.



    CI test + build

    Well, we have CI/autobuild (thanks to Cabalistic).


    Issue tracking with built in conversations

    TDM bugtracker is very good, much more detailed than GitHub tracker yet not monstrous like Jira. And it is filled with lots of useful history :D


    Commit history / insights

    SVN blame and SVN log is enough for history. Insights... is it the metric like who made more commits and which months was more "productive" ? ?



    If you're like me, you probably enjoy following project activity on github, reviewing commits, contributing to issue discussions and occasionally jumping into code with pull requests

    That's the reason why I suggested creating an official mirror of source code repo on GitHub. And don't forget about "increased visibility", which is also very important.



    I say let Alberto have a crack at setting everything up on github, including build tools and then do a show and tell for the community, so the community can decide whether we want to make the official switch.

    It would be a waste of time. There is already one such repo by Nbohr1more, and it gets constant warnings about its size (BTW, I have a reduced git repo with third-party libs excluded from history). Moreover, I think it is updated only manually now.

    In order to set up a mirror, collaboration with our admin is necessary to provide automatic updates. Just converting the repo and putting it on GitHub would be useless.

    • Like 2

  15. I have created an issue: 5079

    Also, hopefully fixed the problem in the latest revision of code SVN. Grayman, I guess it is trivial for you to build an executable yourself?


    Here is what remains to be done here:

    On 12/24/2019 at 11:18 AM, stgatilov said:

    I think the main question to Duzenko is: why is background thread doing anything at all?

    I recall you said that the background thread is used only when cvar is enabled. I don't see suspicious cvar values in the config file.

    UPDATE: I think first of all background thread should be fixed: it must be trivial to validate that it does absolutely nothing or is not spawned at all.

    I recall that I tried to validate that background thread does nothing under cvar when I looked through related commits. But I failed to verify it, since the place where requests are put into thread's queue has not check for the cvar. Then I even put a breakpoint into the background thread and noticed that it loads a few images. I regret I did not raise alarm immediately ?

  16. I'm afraid newer version of the library won't help. Also, changing to a newer library is not an option.

    I think minizip allows to have several unzFile handles into one pk4 file and use them in parallel. In fact, D3 code already does that: since you can open 10+ files in one Zip and store them anywhere, they duplicate unzFile handle using unzReOpen. Therefore right now only opening a file has to be guarded --- reading it is already OK, as long as you don't share the particular file between threads (which is logically wrong anyway).

  17. I have tried to add critical section over the whole OpenFileReadFlags and OpenFileWrite methods. I have run 45 repeats without a crash already. So I think it is a good fix to lock a mutex in most of the methods of idFileSystem, and allow only simple reading/writing concurrently.

    Meanwhile, I'm not certain which locking mechanism should I use. There is Sys_EnterCriticalSection which works with globally numbered sections, and Sys_MutexLock which can work with locally created lock. The global numbering across locks allows to avoid deadlocks by checking that locks are taken in order of increasing number in all possible call stacks. However, if there are many places to have locks, then this list of critical sections can turn into a mess pretty soon.

  18. Quote

    Can you run ... loopped in game console automatically?

    You can create a .seq file, where just write your console commands, one per line. All lines without special character (@  or $ or #) are passed directly to game console. Then you can execute this seq file by running console.py mysequence.seq. It will not repeat by itself,but may be useful. Besides, you can always copy/paste commands in the .seq file ? Or you can use any utility you like to repeatedly run console.py mysequence.seq (batch/shell/python).

    Note that you can also "call" existing .seq-files like @file mysequence.seq. Sometimes it is convenient for composability. So you can start console.py without any argument, then call your script from there --- the console won't close automatically after the script is over.

    Anyway, the simplest way to repeat same script over and over again is to execute repeat.py mysequence.seq. Basically, it simply executes console.py mysequence.seq 100 times in a row, checking for error code every time (to stop on crash). No need to copy/paste or use batch :D It does not restart TDM by itself --- I had to add "exit" command to make it restart.

    UPDATE: It turned out that console.py does not finish automatically when a script is over. So you have to add $sys.exit(0) or @finish at the end of your script to make it work with repeat.py.

    P.S. I don't know if builtin game console supports repeat. Maybe it does. Maybe you can add support for it like: repeat "disconnect; purgeimages; loadgame" 100 (easy to do if quoted text is passed as single argument).

  • Create New...