stgatilov

I think steadily cleaning the code is the most probable answer. The first thing to do will be remove all the dirty hacks from the code. Especially SourceHook hacks

BTW I committed screenshot changes to trunk with jpg as default choice.

Ehm... First of all, I do not intercept the savegame screenshots because I only intercept everything written to files with names "*screenshots\shot?????.tga". And second: savegame screenshots are two times smaller than your D3 resolution, so they take 4 times less space. Still a lot though...

Soon I'll commit TDM version which saves screenshots as jpg-s. I hope . As for uploading images: I seldom use such services and usually ImageShack is enough for me. But today I noticed that ImageShack downsized 1280x1024 screenshots I wanted to show for quality comparison. So I decided to try min.us. It works perfectly, thanks to min.us owner!

Here are the missing screenshot examples: (Do not forget to click on the image if you want to see the original 1280x1024 version)

Did you read the update at my first post? jpg is almost 3 times smaller than png and saves significantly faster (2-3 times). The intention to save the bit-exact image heavily hirts png size and performance. I think it's inherent in lossless compression. I think I'll change my vote from png to jpg (after testing).

Yes, there is a CVAR so each user can set his own format locally. Yes, I believe that jpg quality can be set to 90% (DevIL should support such things). OK, I'll take some screenshots in Heart with 1280x1024 resolution in various formats and measure time + size.

@Tels suggested to change the image format of screenshots taken in TDM. Right now (in my working copy) screenshot format can be set to any of the variants you see in the poll. Other image formats can be added easily. The question is: what screenshot format should be used by default? tga: This format is default now. It is uncompressed. The only drawback is that it is not very standard (compared to bmp). bmp: This format is uncompressed, so the size of screenshots is the same as for tga. png: Standard format for web images, uses lossless compression. The time to take screenshot increases, but size is less. jpg: Standard format for photos, uses lossy compression (can blur images). Size should be even less than for png. UPDATE (Heart indoors 1280x1024): tga, bmp: time = almost instant, size = 3.75 Mb. png: time = 1 sec, size = 1.0 Mb. jpg: time = about 0.5 sec or less, size = 0.37 Mb (don't know exactly what the quality is, but almost surely it is default DevIL value 99%). UPDATE: All the screenshot changes are already in trunk. The cvar is "tdm_screenshot_format" and is equal to "jpg" by default.

CPU prefetchers take data from L1 cache. To get there the data must go through path: (hard drive) -> RAM -> L2 -> L1. And I suppose that RAM -> L2 is bottleneck (and it is MUCH slower than prefetching unaligned data you mention). For example, this code can be greatly accelerated by SSE because it works with the same 4Kb of data many times: for (int i = 0; i<20<<10; i++) for (int j = 0; j < 1<<10; j++) sum += arr[j]; But this code won't work faster with SSE because the huge 80Mb array is accessed once: for (int i = 0; i<20<<20; i++) sum += arr[i]; So I mean that everything might be slow because of small RAM bandwidth. In case of TDM small chunks of memory allocated in the heap are accessed in random patterns. So the effective bandwidth may be even less. By the way I have intel CPU, so I don't know how slow idSIMD is on AMD processors. Perhaps optimizing SSE can improve performance for AMD users. Anyway I wish you good luck!

Forgive me for my pessimism, but I heavily doubt that your work would be useful. I used to play with SSE in TDM and concluded that SSE did not play important role in TDM performance nowadays. First of all, Doom3 engine uses its own private (statically linked) version of idLib. So your changes will only affect TDM code which uses SIMD code rarely. Second, I tested TDM with SSE completely off (there is a cvar to disable SIMD optimization in both idLib instances). The performance was the same as with SSE enabled. I believe that most of the today TDM is RAM-bound (including Tels's ModelGenerator) and rewriting code in SSE won't help performance even a tiny bit. SSE won't increase performance of linear algorithm unless the data is loaded from cache (which is unlikely). And the last idea is about writing SSE code in asm. IMO it is VERY bad decision . You'd have done everything in intel SSE intrinsics (MSDN page). They are very portable (MSVC, GCC, ICC, SunStudio and whatever else). They save vast amount of coding time. And I think that by using them you can get better performance because compiler will handle register allocation (it knows this topic much better than humans). Note that ID decided to use hand-coded assembly instead of intrinsics only because they started coding Doom3 on MSVC6 compiler (13 years old now) which had very bad intrinsics support. Modern compilers have become much better.

nbohr1more, I don't understand the problem here. AFAIU, there is a matrix (table) which has size MAX_ENTITIES x MAX_LIGHTS where there is a record per each entity per each light. Since you simply cannot have more than MAX_ENTITIES entities and more than MAX_LIGHTS lights, why do you need to increase the size of the table? What the hell this table is? What is it used for? Is it internal data structure of the renderer? If you want to know how the rendering is done try to type "r_logfile 1" in console. You'll get huge list of GL calls, annotated with texture filenames and names of rendering phases. P.S. And what do you expect to hear from me?

Why do you need open source engine? Why can't you use the game information from the TDM game part which is open source? Also you need to capture player control, but I think it is also doable without Doom3 source. The game source is stuff like rendering and the like. What do you need there? By the way: some simple hacking of the closed source code can be done. For example, I managed to create a library for bot writing in crimsonland: And here is the bot from Wide Siberian Open programming olympiad connected to this library playing crimsonland:

I played it. Have not noticed any stealth elements . The faster you kill enemies - the more points you get and the sooner you can upgrade. That's why slow sneaky variant is not useful there. Tremulous is mostly action game - the game about running and jumping around . And there are usually several gurus who build the base (if newcomer tried to build a base that would be a disaster).

What do you mean by "data is cached"? Of course loading the same files from hard drive becomes a bit faster the second time, but still. 1.25Gb of data is loaded during starting the new game (measured with process explorer). No cache can handle this huge amount of data! Even if you load the second/third time the hard drive work is still very slow. I run it again with profiling disabled and it got about 200 sec. What does it mean? Nothing It means that hard drive is bottleneck. Do you have P2P/torrent clients enabled? Turn them off and you'll get less start times. Also you may try to start a copy of 4Gb file somewhere on the drive and start a new map in TDM at the same time. You'll likely get much slower loading. It is so. Loading thousands of files more than Gb total with random access patterns is guilty for big mission load times. Decrease size of data loaded. Or rewrite Doom3 pk4 loading routines. Probably you'll get better start times. That's much more interesting! 4ms per entity is long. Can you name any use case which leads to lots of pure entity spawnings in some times? I can start profiling easily for it then if you like. (I do not believe profiling results unless they are accumulated for several seconds of runtime ). Fresh trunk version! Yes, your changes are already used

I think you are optimizing the wrong thing. I started a new game in heart map on max difficulty and it took 287 sec according to console message. And most of that time was spent on reading from hard drive. Profiling shows that 6% of CPU time was consumed by Doom3.exe and 0.5% of CPU time was consumed by gamex86.dll. 82% of CPU time was not used at all. Most of the time was spent on reading all the pk4-s from the hard drive I think, unpacking them and etc. If you want to decrease this time you should think about rearranging files inside pk4 probably (though I think it is not an option). Using better coimpression on assets is one of the proper ways to decrease map load time. If you need to decrease entity spawning time then you should profile another place where entity spawning is a bottleneck.

Here they are.

The map is "Heart of Lone Salvation", it was released long time ago . You can download it from here.

I think yes. He is likely one of the guards from the jam. Maybe debugging a save could help? You won't see how he got into such state, but you'll be able to see what is he doing and what his state is. Yes, that's exactly the place. I'm not sure that I can reproduce it. It was rather late in the game - a lot of events happened already. Moreover, I was more like running through the FM killing everything I meet than playing normally...

Both images are taken at the same place. You can see the doorway from the first image on the left wall in the second image. The door is between the corridor and kitchen. There's really another opened door to the left, but they don't use it. The tough (why has he so much HP?) guy in white is CooksAssistant I think. He is the one guilty for being stuck. The console repeatedly shows message: "AI MoveStack contains more that 100 moves! (CooksAssistant)". I waited for several minutes with no result. All in all 4 guys are involved: CooksAssistant in white: stuck, does not move; servant closer to me: tries to run away, runs like mad; two guards on the other side: I guess they are searching for me, but cannot pass through the cook. I tried to kill servant running around and toggled lantern on. The remaining three guys are stuck again. The guards are clearly in combat state. Saves are on FTP: "Guards Stuck", "Guard does not attack"

Tried to play through heart on new SVN branch. I met a few problems with AIs... The first one is minor: guards got stuck forever in a doorway. The second one is more serious. The melee guard does not attack me - just runs into me. I've seen an entry in bugtracker entries with similar description...

Yes! Not completely without distortion, but it will become negligible. And you'll be able to set 360 degrees horizontal FOV if you like

Yeah, setting viewports to 1/3 of framebuffer for each shot is what I meant. Perfectly accurate results? Do you mean that monitors look like a window to parallel TDM-world? Complete angular equivalence?... I've seen panorama pictures on flat monitor. They are far from accurate, but very cool . It is not necessary to be completely accurate to get cool results. I think Bikerdude should say the final word like "yes, that's interesting" or "no, it's not worth efforts" about the triple-view rendering discussed here. By the way, can you get aspect ratio 5:1 in windowed mode?

Running a game in 5:1 aspect ratio is not a good idea I think. You'll have small vertical FOV and close to 180 degrees horizontal FOV which will be stretched a lot on the sides (I think it's the fisheye effect mentioned). Maybe it's better to stop on the vertical monitor orientation case? I think the aspect ratio restriction can be deleted (with hacking or not). Do you want it so badly ? There is another cool idea with three horizontal monitors. The screen surface can be bent around you - you can actually SEE your monitors in FOV more than 180 degrees this way. Moreover, if you find another 20-30 monitors you can surround yourself with them . Imagine that you need only to turn your head back to see if there is a guard there It is much easier to buy VR googles though... By the way, do you like the idea of running TDM with 300 x 60 FOV degrees? I think it can be achieved, but with high cost. The SDK code can be fixed to render three views (separate view for each monitor). The hacky part is to redirect output to separate monitors. Maybe it can be done by setting separate viewports for them.

I think the issue is about eliminating pipes from lightgem shot receiving. And using glReadPixels directly from SDK was intended as a possible solution. Right now the state is the following: Firewall issues would be fixed in 1.05 by eliminating pipes (hook WriteFile instead). This relevant code is already implemented and is working right now (in branch), but it is considered risky to ship it in 1.04. And everyone is busy testing a lot of other things EDIT: by the way using glReadPixels directly wouldn't make performance much better I think.