Converting model .tgas to .dds

[New Horizon]

For me locally, I'm currently trying to determine the exact cvars to set so that I keep my current resolution, but perhaps bump my texture sizes down a notch (if anyone knows of such, help a guy out).

 

There is the 'downsize' options in the doomconfig. You set them to the smallest you want your images downsized to.

[Springheel]

Interesting. Is that built into the video settings? Might be worth adding that option if it isn't already part of "Video Quality".

[Springheel]

Alright I just found a tool with the source available (it's really small) that converts tga and bmp to DDS (DXT3 w/Mipmaps), I just converted 27 megs of textures down to 4.5 megs.

 

I just tried this on the mainmenu images, and it would only covert five of the thirty or so images. Some of them it just refuses to convert for some reason. Is there some trick to this?

[greebo]

Guys, please plan this step out carefully - just saying. I was there at the last reorg, doing a significant amount of work (it's safe to say this without patting myself on the shoulder) and this is gonna take away a lot of energy and concentration (plus time).

 

Also, whoever is going to be involved with this, please study the wiki articles well. There is already a lot of information sitting there, and we're discussing things here that have been nailed down in the past (like the addnormals issue).

 

Whoever is going to convert these, please make sure you know what you're doing and you know about DXT1/3/5 compression. Otherwise you're in danger of wasting work and time.

 

DDS diffuses and speculars without alpha channels: DXT1. Everything with alpha channels use DXT3. Use ATI's compressonator, it can be used from the command line too, if this is really necessary (I believe it's not necessary, because you can Ctrl-Click-Select the files that need to be converted).

 

You won't be able to pull this off with just a script.

 

DDS textures can not be used with addnormals() or other mapexpressions. This is important for the models like the gargoyle or other stuff which is using "stock" darkmod textures/normals plus mapexpressions. You need to check whether models use this kind of trickery before you can convert the textures. Watch out for dependencies of models on default darkmod textures.

 

Normalmaps need to be compressed with ATI's compressonator. GIMP is not compatible, as far as I know. You'll need to test it if newer versions of GIMP support saving of DXT5 RXGB normalmaps. This is a D3-specific format, so you really need to make sure.

 

I have strong doubts whether this can be pulled off before release. If you ask me, postpone this after release and plan this properly.

[Springheel]

DDS textures can not be used with addnormals() or other mapexpressions.

 

Were you aware of the dds/addnormals folders in the default D3 folders, greebo? It appears that they used some kind of .dds compression that was able to work with addnormals. I don't know what kind of .dds it is, however...when I try to load it I get a garbled image.

[greebo]

Were you aware of the dds/addnormals folders in the default D3 folders, greebo? It appears that they used some kind of .dds compression that was able to work with addnormals. I don't know what kind of .dds it is, however...when I try to load it I get a garbled image.

That's because they're in DXT5 RXGB format, this is not something you can open in Photoshop. You'll need to use ATI's compressonator (DarkRadiant could open these too, but DR has no image viewer, maybe I should write one).

 

At any rate, I didn't notice that folder before, that's definitely a good hint. This looks like exactly the folders which get produced by the D3 image_write* console commands. It seems like DDS textures which are included in addnormals() and heightmap() expressions are directly sought in these types of folders, where the "precompiled" versions are stored.

 

To produce these folders, one needs to have the TGA version available, run D3, then type image_writeNormalTGA and D3 produces these DDS files. This will "bake" the addnormals/heightmap/makeIntensity/whatever mapexpressions into these special DDS folders, where they are picked up again when D3 is run on lower quality settings.

[SneaksieDave]

There is the 'downsize' options in the doomconfig. You set them to the smallest you want your images downsized to.

Interesting. Is that built into the video settings? Might be worth adding that option if it isn't already part of "Video Quality".

Yep, and agreed, it should be there. I tried this out today with image_downsize 256, and while it looked... well, like 256X256 images of course, all of saintlucia fit into ~500Mb of RAM! And my framerate was almost double normal in many spots. This (and related commands) is absolutely awesome if you ask me, and speaks to the flexibility of Doom3. If someone makes a gargantuan map, and you simply haven't got enough RAM to run it, use downsize, and images are capped at the set size. Wow, what other game allows users to do that??

 

I have strong doubts whether this can be pulled off before release. If you ask me, postpone this after release and plan this properly.

Agreed that that's probably the most sensible thing to do.

 

Now, the big news (on my end at least), which I hope does not derail the current compression effort, because it is very important IMO.

 

So I went through today and tried a bunch of cvars (still more to go) and wouldn't you know it, I was bitten in the ass by the bug where Doom3 overwrites (or fails to write?) your doomconfig and serial keys; several of us have mentioned it so chances are we've all hit it now and then. At some point, I got that problem (before setting them read only) and guess what happened as a result, unknown to me? Goddamnit. It reset image_useNormalCompression back to 0! Part of the reason I've been having memory problems with saintlucia is that setting! When set to 2, as it should be (we hear that often from NH), all of saintlucia fits into my RAM! No texture thrashing. I have to test more (I had to run out) but I think the 'portal flood' may have actually been a 'texture flood' -- not that it was thrashing at the spot, I waited plenty; but instead that that scene contains more than 128Mb of textures (my card's capacity). When in RAM, it slows down a bit, but when on HD, it drops to 1-2 FPS.

 

Some obvious points I'd like to state, lest this derail compression efforts:

 

1. So Saint Lucia does fit within 1Gb of RAM. (I know, I know... you already knew that). That's great. But of course, larger maps (and saintlucia isn't that big) will simply run into the problem again.

 

2. While that can't be prevented at some point, as someone's always going to make a map that's too big for someone, whatever we can do to help, we should. And that probably means using compressed textures as planned (separate from my fouled up setting).

 

3. I completely agree about making the image_downsize settings available in the menu, AND, possibly making image_useNormalCompression blatantly obvious as well; I had no idea this cvar made this much difference. On my system, it removes >250 Mb(!) of memory load from saintlucia.

 

So in summary, nothing was wrong (hardware) on the system, this map can fit in 1Gb, and a damned Doom3 loss/reset of config bug provoked the whole mess. Sorry for the hubbub, and an honest thanks for pushing me and helping to figure out the problem.

 

Of related interest, I discovered that it's actually more demanding to open saintlucia in DR than TDM. Perhaps that's not surprising, but I just assumed it'd side with Doom3, because it's the game engine, sounds, AI, lighting, etc. This brings to mind a question, and a task. The question, to the DR devs: is there any chance DR could make use of similar texture compression? I imagine it's bogging down loading all of the normal maps, and they can't be shrunk down with editor versions. If it's worth a tracker entry (even for future), please speak up. And the task, related to that: I'm going to (I'll see how far I can get before I run out of steam, but I'm personally motivated for this) go through all of our brush textures and make sure they all have editor versions (JPGs). That should cut memory issues quite a bit (though not as much as compressed normalmaps).

[Crispy]

Yikes. Well I'm glad that mystery is solved.

 

If Doom3 can randomly reset configurations like that, it makes this feature I suggested earlier (around TD) much more important. If that had been implemented, this whole kerfuffle would never have happened. Do we have any volunteers to implement this feature? I suggest one of the programmers who isn't as familiar with the SDK takes this on, as it's an ideal "getting your feet wet" task.

 

(Pity that these cvars aren't defined in the SDK code, otherwise we could set their default values to the ones we want. )

 

I imagine image_downsize puts an extra initial load on the CPU as well as decreasing quality - so if we do put it on the menu, it should come with a big fat red warning that it should only be used in cases of HDD thrashing on large maps.

[Springheel]

It reset image_useNormalCompression back to 0! Part of the reason I've been having memory problems with saintlucia is that setting! When set to 2, as it should be (we hear that often from NH), all of saintlucia fits into my RAM! No texture thrashing

 

I feel like I missed that discussion. What is the effect of this setting, other than speeding things up?

[New Horizon]

image_usecompression 1 forces the compression of diffuse textures, and image_usenormalcompression 2 forces the compression of normals. This will increase load times of course.

 

image_downsize cvars, simply cause textures to be scaled down. You can enable this to work with or without compression enabled.

 

image_useallformats 1 makes the engine ignore quality settings and use whatever you throw at it.

 

I would still recommend using dds textures for everything though. They're smaller in size, and won't increase load times like these settings will.

[SneaksieDave]

Basically, with a setting of 0, I was getting no compression of normals in game, at all. So every one of those huge 4Mb textures was taking whatever real size it occupies. With it set to 2, more than 250Mb of memory demand was lifted.

 

Edit: Ninja'd.

[Springheel]

So this will decrease 'stutters' where you suddenly encounter a bunch of new textures, but will increase the load time for the map (which is already at about 5 minutes for me)?

 

And changing textures to .dds only shortens map loading time? Or would it also affect map performance?

[AluminumHaste]

So this will decrease 'stutters' where you suddenly encounter a bunch of new textures, but will increase the load time for the map (which is already at about 5 minutes for me)?

 

And changing textures to .dds only shortens map loading time? Or would it also affect map performance?

 

I was going through the config files for doom 3 and darkmod and couldn't find alot of these cvars so I added the relevant ones. You should all go through these and change your settings appropriately.

image_useCache [0,1] - If set to 1, uses background loading to cache image information. This may not necessarily improve framerate performance, but it does assist in smoothing out frame rates and reducing loading pauses. Note that the image_cacheMinK (see below) size must be raised from its default otherwise the game will crash when using this setting. Once this setting is enabled, the actual cache value is set in the image_cacheMegs setting below.

 

image_cacheMegs [Megabytes] - Determines the maximum amount of system memory to allocate to temporary loading of full-sized precompressed images if the cache is enabled by setting image_useCache to 1. Note that since this caching only applies to precompressed images, using the Ultra Quality setting will render this setting useless, since at Ultra quality no texture compression is used. Since a cache is only a temporary holding area, and not the place where the entire game is meant to reside, do not raise this value to something extremely high as that will simply reduce the available memory for the rest of Doom 3. Try a value of 128MB for those with 512MB of system RAM, and if you have 1GB or more of RAM, you can try a higher value like 196MB or 256MB for example. Do not automatically set this to half your system RAM or some monstrous amount.

 

image_cacheMinK [KiloBytes] - This setting determines the minimum size in KB for precompressed image files to be loaded into the cache. To make sure most image files can be loaded, and to also prevent a system crash when enabling caching, select a value like 3072 to start with. Raising this setting may improve caching, but it may also hinder caching if most textures are below the size you specify, so don't just raise this value for the hell of it.

 

[Graphics Tweaks]

 

r_brightness [value] - Determines the overall brightness for the game. You can adjust this using the slider in the main options menu (See In-Game settings), however if you want to fine tune it you can do so here. Has no performance impact.

 

r_gamma [value] - Determines the gamma level (essentially a form of brightness) for the game. Raising this value too high can make the game image appear washed out. Has no performance impact.

 

r_displayRefresh [Hz] - Determines the refresh rate used when Doom 3 is launched. You should already be getting the maximum refresh rate possible on your monitor when playing Doom 3 through the use of a refresh rate fix (See my ATI Catalyst Tweak Guide for a link and instructions for both Nvidia and ATI users), however if you want to set this manually, determine the supported maximum refresh rate on your monitor for the resolution you've chosen for Doom 3 (e.g. I run Doom 3 at 1280x1024, and my monitor can do 85Hz maximum at this resolution), and set this variable to that value.

 

r_multiSamples [0,2,4,8,16] - This setting determines the level of Antialiasing samples used. This can be set using the in-game menus (See In-Game Settings section). The numerical values here are 0 which is no antialiasing, 2, 4, 8 and 16.

 

com_videoRam [MB] - Shows the detected amount of video memory on your graphics card in MB. If this value is incorrect, set it to the correct value to improve performance.

 

cm_backFaceCull [0,1] - If set to 1, removes all polygons behind the direction the player is facing - that is, all the things the player cannot see. This can improve performance, but if you experience any visual anomalies set this back to 0. Note it's cm_backFaceCull, not com_backFaceCull.

 

com_purgeAll [0,1] - If set to 1, discards all data between level loads. Can increase loading times noticeably but may help reduce in-game pauses.

 

com_machineSpec [-1,0,1,2,3] - Shows the currently detected machine specification level, with the choices being -1 - Not detected, 0 - Low quality, 1 - Medium quality, 2 - High quality, and 3 - Ultra quality.

 

com_fixedTic [-1,0,1] - If set to 0 (the default) Doom 3's ticrate is capped at 60FPS, meaning the game framerate cannot exceed this, except when running a timedemo. If you want to remove this cap, set com_fixedTic to -1. Setting it to 1 removes the cap, but causes the game physics to go out of sync. Removing the FPS cap doesn't necessarily improve performance, and can lead to more variable framerates. Note that com_fixedTic -1 doesn't work as of the latest patch.

 

setMachineSpec - Detects your system hardware and sets the value for com_machineSpec (See above).

 

execMachineSpec - Executes all the relevant config files and sets variables based on the machine specifications provided by the com_machinespec variable. For example, if you raise your machine spec to 3 using "com_machinespec 3", then running the command "execMachineSpec" will change all your settings and config variables to the ones appropriate to a machine suited for Ultra quality performance (i.e very high end).

 

r_vertexBufferMegs [Megabytes] - This setting controls how many MB are allocated to the vertex buffer. Increasing this value will see a performance improvement on most systems, so try increasing it to 48MB or 64MB.

 

r_orderIndexes [0,1] - If set to 1, reorganises indexes to optimize vertex use. This improves rendering performance.

 

r_usePortals [0,1] - If set to 1, uses portals to perform area culling, otherwise everything is drawn. Setting this to 1 provides the best performance.

 

r_useOptimizedShadows [0,1] - When set to 1, uses the dmap generated static shadow volumes, which provides optimal performance.

 

r_lightScale [value] - All light sources are multiplied by this value. Increasing this value increases the intensity of all visible light sources. Basically it makes light sources brighter the higher the value.

 

r_skipParticles [0,1] - If set to 1, this disables all particle effects, including smoke, fog, steam etc. This can dramatically improve performance in areas with these effects, however it can reduce realism greatly as well.

 

r_skipDiffuse [0,1] - If set to 1, disables diffuse lighting. This improves performance, but also darkens areas where such lighting is used since no lighting is used to replace the removed diffuse lighting.

 

r_skipUpdates [0,1] - If set to 1, lighting is not updated, meaning all lighting will become static (fixed direction and intensity). This can improve performance significantly, but reduces realism.

 

r_skipDeforms [0,1] - If set to 1, any material which can be deformed will remain in its original state.

 

r_skipFogLights [0,1] - If set to 1, disables all fog lights which can improve performance.

 

r_skipTranslucent [0,1] - If set to 1, disables translucent rendering, which can improve performance (but reduce realism) in such areas.

 

r_skipInteractions [0,1] - If set to 1, disables all light interactions with surfaces, effectively making almost all surfaces black.

 

r_skipMegaTexture [0,1] - If set to 1, uses only the lowest level images. This improves performance at the cost of lower image quality for some surfaces.

 

r_skipGuiShaders [0,1,2,3] - If set to 1, this removes all Graphical User Interfaces (GUIs) in the game (i.e. computer terminals, security screens etc.), if set to 2, GUIs aren't drawn but they can still be interacted with, if set to 3 they will be drawn, but cannot be interacted with. These settings can improve performance but obviously reduce playability.

 

r_useShadowCulling [0,1] - If set to 1, tries to remove shadows from partially visible lights. This provides optimal performance.

 

r_useLightCulling [0,1,2,3] - Determines the light culling method. 0 is no culling, 1 is box culling, 2 is the exact clip of polyhedron faces, and 3 also includes areas. This should be set to 3 for optimal performance. Other methods can reduce visual anomalies with lighting but will reduce performance.

 

r_useCulling [0,1,2] - Determines the object culling method. 0 is no culling, 1 is sphere culling and 2 is sphere and box culling. Setting this to 2 provides optimal performance, but other methods can be tried to reduce visual anomalies.

 

r_renderer [best, arb, arb2, cg, exp, nv10, nv20, r200] - Determines the rendering path to use. Some renderers can't run on certain hardware - for example cg is an Nvidia rendering language and can't run on ATI cards. You can experiment with these to see which is best on your system, as I haven't had the chance to test these on various hardware.

 

r_ignoreGLErrors [0,1] - This setting should be set to 1, to ignore OpenGL errors. Most people will have one or two OpenGL errors throughout the game, and these don't seem to affect gameplay.

 

r_znear [value] - This setting determines the distance at which z-buffer clipping occurs. The default value is 3, and lower values can result in graphical anomalies for distant objects but improve performance, while higher values may help reduce any z-buffer graphical anomalies, but may reduce performance.

 

r_useInfiniteFarZ [0,1] - If set to 1, this uses the no-far-clip-plane trick which provides optimal performance.

 

r_useStateCaching [0,1] - This option should be set to 1 for optimal performance, as it caches OpenGL state changes.

 

r_useIndexBuffers [0,1] - If set to 1, uses ARB_vertex_buffer_object for indexes. This can actually reduce performance, so set to 0.

 

r_useVertexBuffers [0,1] - If set to 1, uses ARB_vertex_buffer_object for vertexes, which improves performance.

 

r_useCachedDynamicModels [0,1] - If set to 1, uses cache snapshots of dynamic models which improves performance.

 

r_useTwoSidedStencil [0,1] - If set to 1, optimizes the processing of stencil shadows.

 

r_useTurboShadow [0,1] - If set to 1, uses the infinite projection with W technique for dynamic shadows which improves performance without a noticeable drop in image quality.

 

r_useShadowVertexProgram [0,1] - If set to 1, does the shadow projection in the vertex program on graphics cards with this capability. If you set this to 1 and experience problems or a crash, it is because your graphics card does not support this functionality.

 

r_useShadowProjectedCull [0,1] - If set to 1, removes polygons which would fall completely inside shadows, optimizing performance.

 

r_useTripleTextureARB [0,1] - If set to 1, this allows graphics cards with 3+ texture units do a two pass instead of three pass which improves performance. Set this to 1 and if you experience problems, your graphics card does not support this function.

 

r_useConstantMaterials [0,1] - If set to 1, this setting uses pre-calculated material registers if possible, improving performance.

 

r_useNV20MonoLights [0,1] - If set to 1, this option uses optimized rendering for mono lights.

 

r_mode [-1,3,4,5,6,7,8] - This setting determines the resolution, as set in the in-game menus under Screen Size (See In-Game Settings section). The mode values are 3 = 640x480, 4 = 800x600, 5 = 1024x768, 6 = 1152x864, 7 = 1280x1024 and 8 = 1600x1200. Note, you can use -1 for r_mode, and this tells the game to use the values for r_customHeight and r_customWidth to set screen width and height - this is useful for allowing things like widescreen mode.

 

r_aspectratio [0,1,2] - This setting allows you to set the aspect ratio of the image, which is mainly used for displaying on TVs. The options are 0 = 4:3 (normal), 1 = 16:9 (standard widescreen), and 2 = 16:10 (nonstandard).

 

r_useLightPortalFlow [0,1] - If set to 1, this uses a use a more precise area reference determination for lighting. Setting this to 0 improves performance but may result in graphical anomalies.

 

g_projectileLights [0,1] - If set to 1, allows dynamic lights for projectiles. Setting this to 0 can improve performance in combat.

 

g_doubleVision [0,1] - If set to 1, shows a blurry "double vision" effect when taking damage. Setting this to 0 removes this effect and can improve performance when taking damage.

 

g_muzzleFlash [0,1] - If set to 1, shows a flash from the muzzle of weapons that are firing. If set to 0, this effect is disabled, which can improve performance.

 

g_showBrass [0,1] - If set to 1, visible shells will be ejected from weapons. If set to 0, no visible shells will be ejected, and performance may be improved in heavy combat.

 

g_showHud [0,1] - If set to 1, this enables the Heads Up Display (HUD). If set to 0, this removes the HUD from view, which may give a minor performance improvement. This command is most useful for taking "clean" screenshots (i.e. without the HUD).

 

g_showPlayerShadow [0,1] - If set to 1, the player will cast a shadow when standing near lights. If set to 0, the player casts no shadow, which improves performance.

 

g_nightmare [0,1] - If set to 1, Nightmare difficulty level can now be selected on the main menu screen. If set to 0, this option is disabled. Normally players must complete the entire game before Nightmare mode is enabled.

 

g_bloodEffects [0,1] - If set to 1, full blood effects (blood sprays, splattering etc.) is enabled. If set to 0, this is removed, which may yield a small performance improvement in heavy combat.

 

g_skipViewEffects [0,1] - If set to 1, removes various view-related special effects like the damage effects (double vision). This can improve performance.

 

g_fov [angle] - Specifies the angle for the Field of View (FOV). The wider the angle, the larger the field of vision for the player, giving a "fish eye" view in effect, however performance will be reduced. The smaller the angle, the higher the performance. The default of 90 is recommended, but players can try 85 which is a value used in some other games and will gain a slight boost in FPS. In multiplayer, g_fov can only be changed between 90 and 110.

 

g_skipFX [0,1] - If set to 1, disables certain special effects (I'm unclear on which). This can improve performance.

 

g_skipParticles [0,1] - If set to 1, disables certain particle effects (does not include smoke, dust, fog effects etc). This can improve performance, particularly in areas which use these effects to a large extent, but realism will be reduced.

 

image_usePrecompressedTextures [0,1] - When set to 1, all textures will be precompressed, resulting in less video memory being used and hence smoother performance on most machines. There may be some minor compression artifacts on textures, but these are hardly noticeable. If set to 0, this setting will force realtime compression on textures which results in reduced image quality and performance. Note that if this setting is set to 0 and the image_useCompression setting is also set to 0, uncompressed textures will be used for highest image quality, but performance will suffer and you will get more small pauses depending on the amount of Video RAM on your graphics card, so on balance this should be set to 1.

 

image_useNormalCompression [0,1,2] - Specifies the compression method used for textures. 0 disables normal map compression for the highest image quality for textures, 1 is 256 color compression (if available), which gives the best performance but provides the most compression artifacts, while 2 is rxgb compression. When combined with the setting above and below, setting this to 0 provides optimal image quality for minimal performance hit.

 

image_useCompression [0,1] - When set to 1, compression is used - with the method of compression specified in the image_useNormalCompression variable. Compression provides the fastest performance by reducing the size of all textures and light maps, but there are some (minor) compression artifacts. When set to 0, precompressed textures are still used, but realtime compression is not used if image_useNormalCompression and image_usePrecompressedTexures are activated. This results in better image quality, but slower performance particularly during periods when precompressed textures are not used.

 

image_preload [0,1] - If set to 1, Doom 3 attempts to preload most images in the background, meaning less loading pauses during the game but longer loading times. If set to 0, images are loaded dynamically as needed, meaning more loading pauses during the game. 1 is the recommended setting.

 

image_lodbias [value] - Determines the Level of Detail (LOD) bias for mipmapped images. The higher the number, the less detail is visible on surrounding objects. This can improve performance, but reduces image quality. The default is 0, and values above 1 reduce detail quite heavily, so experiment with fractional values such as 0.2 or 0.5.

 

The ones I found most interesting were the culling ones as those look useful.

[SneaksieDave]

So this will decrease 'stutters' where you suddenly encounter a bunch of new textures, but will increase the load time for the map (which is already at about 5 minutes for me)?

That's what I'm seeing. The map fits well within 1G RAM, but it does take a bit longer to load due to the compression step.

 

And changing textures to .dds only shortens map loading time? Or would it also affect map performance?

DDS are also smaller, already compressed, so they should give both benefits. But, since we don't use DDS normals, that's why the cvar is vital.

[greebo]

Of related interest, I discovered that it's actually more demanding to open saintlucia in DR than TDM. Perhaps that's not surprising, but I just assumed it'd side with Doom3, because it's the game engine, sounds, AI, lighting, etc. This brings to mind a question, and a task. The question, to the DR devs: is there any chance DR could make use of similar texture compression? I imagine it's bogging down loading all of the normal maps, and they can't be shrunk down with editor versions. If it's worth a tracker entry (even for future), please speak up. And the task, related to that: I'm going to (I'll see how far I can get before I run out of steam, but I'm personally motivated for this) go through all of our brush textures and make sure they all have editor versions (JPGs). That should cut memory issues quite a bit (though not as much as compressed normalmaps).

DarkRadiant doesn't compress textures at all. The algorithm is: load bitmap from disk (using any suitable loader it can find), tell the graphics card driver to load it into memory (including mip map generation) and that's it. It is possible to downsize the textures to a certain size before loading them, but we're using 256 pixel editor images anyway already, so there's not much to be gained in non-lighting mode.

[SneaksieDave]

I meant specifically normalmap compression. Since those are the giant files, if they could be compressed*, it'd save memory the same way doom3 does with image_useNormalCompression. Is that feasible?

 

*Alternately, probably a simpler brute-force approach, would it be possible to have a preference/setting where the user says "skip loading normalmaps completely"? They're not terribly important in an editor, and rendered mode is currently lacking anyway, so the most of what they do is eat memory by the bucketload. Performance would benefit greatly from being able to optionally skip them. SaintLucia for instance would fit in <750Mb of memory, instead of 1.2Gb -- no HD thrashing at all. No need for "low" (1Gb) memory systems to work on maps like this in frustrating pieces, because the whole thing can be handled.

[greebo]

The normalmaps are not loaded by DarkRadiant unless you enter lighting mode in the first place. The textures are loaded "on demand" so to say.

[SneaksieDave]

Wow, so even before loading normalmaps, it's using 1.2Gb. Well it's good news at least, that it's not consuming the memory for normals unless you ask for it. So, I take it compression of any type is in fact not a viable request......(?)

 

On a positive note, and at least slightly related to the TGA/DDS discussion, I just confirmed that AI materials can use qer_editorimage params too, so I can add that onto my current task (won't affect the DDS conversion); this should conserve a ton of editor memory, as some of our largest textures are on AI. The effort is all the more important considering the "on demand" revelation above.

[Tels]

Sorry that I reply so late, I managed to cut of my internet+telephone completely (blasted incompatible plug/connectros grumblemumblemutter)

 

Yep, and agreed, it should be there. I tried this out today with image_downsize 256, and while it looked... well, like 256X256 images of course, all of saintlucia fit into ~500Mb of RAM! And my framerate was almost double normal in many spots. This (and related commands) is absolutely awesome if you ask me, and speaks to the flexibility of Doom3. If someone makes a gargantuan map, and you simply haven't got enough RAM to run it, use downsize, and images are capped at the set size. Wow, what other game allows users to do that??

 

 

Yeah I think a "downsize textures" setting with 126, 256, 512, 1024, 2048 and off setting would be useful. It is certainly more easier to tell users to try this setting when they have problems instead of telling them to much with the config

 

Likewise, the "useCompressedNormals" setting should also exposed, that would make such mistakes much more visible

 

Sneaksie, I am also glad we solved that mystery See, I knew your system shouldn't be that slow

 

Oh, and we also need to mention that 1Gbyte of memory is about the minimum your system should have, or troubles might start.

 

PS: I am quite happy to hear you can now play SL!

[Tels]

So this will decrease 'stutters' where you suddenly encounter a bunch of new textures, but will increase the load time for the map (which is already at about 5 minutes for me)?

 

Pretty sure someone already answered this one, but anyway

 

* the CPU will take some time to compress the TGA normalmaps

* however, the _system_ memory usage will be much lower

* and since normalmaps need to be send to the graphic card to be part of the rendering, the _video_ memory usage will also be much lower

 

Basically, if you have enough memory, then it will take a (very tiny) amount of time, and keep more memory for buffers, caches etc. around. Plus your system will swap much less data between system and video memory - which makes things faster, esp. for people with non PCI-Express cards (transfer over AGP bus is much slower).

 

But if you dont have enough memory (like sneaksie) it will prevent your system from swapping things out to HD (and back in when they are needed), and since swapping is _really_ slow, this will make everything faster _alot_. You get the video memory benefit, too, of course.

 

So useCompressedNormalmaps should always be on, as the benefit grossly outweights the small speadhit of the CPU compressing the normalmaps upon loading.

 

And changing textures to .dds only shortens map loading time? Or would it also affect map performance?

 

it affects performance since smaller texture in memory => smaller transfer to video card => less memory on videcard used => more memory for more textures => even less transfer.

 

 

(Almost all the aforementioned points are moot if you have the Ubersystem - 8 Gbyte main memory and 1Gbyte video memory mean that basically everything fits into memory and the compression frees memory that you have, anyway. But most people do not have such systems and we don't need to care for such systems anyway, as they are fast enough

[Springheel]

@Sneaksie: I see you've been doing a bit of work here--if you have time, I'd love to see the effects of changing the menu .tgas to .dds files. Many of them are 1024x1024 .tgas, and thus take up a lot of space. I don't have time right now to go through them one at a time and the batch converter didn't work for me.

 

(note that a great many of them have transparent areas or are partially transparent, so need to be compressed accordingly)

[SneaksieDave]

Oh yeah, some of those are the whopper 4Mb ones. The biggest ones at least should be converted. I wonder if that will increase load times much. Not sure about the transparencies; do we have consensus on that? Doesn't that require some ATI tool, or is any dxt3 enough?

[Tels]

Oh yeah, some of those are the whopper 4Mb ones. The biggest ones at least should be converted. I wonder if that will increase load times much. Not sure about the transparencies; do we have consensus on that? Doesn't that require some ATI tool, or is any dxt3 enough?

 

It will very very probably _decrease_ load times. Compare loading 4Mbyte from disk to loading 512Kbyte (or whatever) - in both cases the result of the load will be used directly, anyway.

[Springheel]

I would *hope* it decreases load time, otherwise there's not much point.

[New Horizon]

Using a physical dds file instead of a TGA file won't increase load times, it will greatly decrease them. The only time you will see increased load times is when doom 3 has to compress every TGA to DDS at load time, when usenormalcompression is set to 2 and when usecompression is set to 1. That's why all our TGA's should be physically converted to dds. Then the only setting we would need to set is useallformats 1.