Trouble Loading RXGB DDS Files

[SneaksieDave]

Do you have pak004.pk4? And does it contain TGA files?

Yes, that and pak001.pk4 both.

 

If my value of normal compression was by install set to 1 instead of 2 - and that is different from the norm (har har), wouldn't that suggest this is all by design (as in, something GF4 gen cards can't do)? Or is that not a consideration?

[SneaksieDave]

So what's the current status/plan? I've got more textures to submit, fixup, and still most of the ones I originally submitted not up yet. I'm itchy, can ya tell? Can we just not compress them for the time being at least, perhaps substituting DDS versions of the normalmaps later, when/if the problem is ever figured out? If it's never figured out, we'd need two repositories anyway - one compressed, one uncompressed.

[OrbWeaver]

I have to say I am extremely concerned about the use of DDS for original textures. While I understand the reasoning as regards the size of the download, and agree with it in principle, the fact remains that exclusively using DDS textures is just something Doom 3 was not designed to support.

 

I think as we go forward we may find that the amount of hackery required to get the game to work properly, when the original TGA images that Doom 3 expects simply aren't there, becomes more trouble in terms of maintenance than its size advantage justifies.

[New Horizon]

I have to say I am extremely concerned about the use of DDS for original textures. While I understand the reasoning as regards the size of the download, and agree with it in principle, the fact remains that exclusively using DDS textures is just something Doom 3 was not designed to support.

 

I think as we go forward we may find that the amount of hackery required to get the game to work properly, when the original TGA images that Doom 3 expects simply aren't there, becomes more trouble in terms of maintenance than its size advantage justifies.

 

I believe it was designed to do it, but it's just not setup to do it. I'm checking out the Prey Demo as it uses DDS pretty much exclusively and the game looks beautiful. I think there is a hesitation in using DDS due to the over compression in TDS...but fear not, if they're done properly...they look exceptional.

[OrbWeaver]

I wonder if its possible to specify a DDS file directly as part of a map stage in a texture, rather than using a dummy TGA path which is replicated with a DDS in the dds/ folder.

[SneaksieDave]

They might be great, when they load. For some users, like the guy making the textures in this thread, they won't even load for whatever reason, and I would simply like to be able to use my textures and get on with it.

[New Horizon]

They might be great, when they load. For some users, like the guy making the textures in this thread, they won't even load for whatever reason, and I would simply like to be able to use my textures and get on with it.

 

I'll look into it tonight and see if I can come up with it. There has got to be a simple explanation for what is happening.

 

I'm cross referencing the prey demo to see exactly how they have things setup.

[Gildoran]

@NewHorizon: OrbWeaver and I have been trying to figure that out, without much luck. I figured out that for some reason SneaksieDave had image_useNormalCompression set to 1 instead of 2 (and I don't think he's been tweaking his cvars, so his installation must have set it that way for some reason), making his copy unable to read the RXGB compression format that's usually suggested for normalmaps in D3. However, what's strange, is that when it's set to 2 for SneaksieDave, he can't read the other compression format (256 color palette) used by D3, but everybody else can. Orbweaver suggested the good idea that perhaps when image_useNormalCompression is set to 2, D3 uses the original TGAs to construct new DDS files in RXGB format, and SneaksieDave isn't able to load the vanilla D3 textures because he doesn't have the TGA versions of them... but SneaksieDave seems to have the original TGAs, which isn't consistent with that idea. ...so I'm not sure what to think at this point.

 

One thing we could do is just go full-out with RXGB normal compression, encoding all TDM textures that way, and say that vanilla D3 textures aren't supported by TDM. This would work with SneaksieDave's installation when he has image_useNormalCompression set to 2. However, I still find it worrisome that there's this unexplained anomaly of him experiencing different behavior from everybody else... could there be people who can't read RXGB at all? On the bright side, if we do that and worse comes to worse, we wouldn't have to modify anything in the darkmod repository to fix it... we could just zip up the hires repository and provide it as an optional PK4 for users who have trouble with the RXGB format. (though it might be a good idea to shrink the images in the optional PK4 to 512x512)

 

@SneaksieDave: BTW, I just remembered something I needed to ask you... With image_useNormalCompression set to 2, can you load our TDM textures that don't have any DDS files?

[SneaksieDave]

Holy macaroni! As the guy (arguably representing a small portion of users): just going compression 2 and forgetting D3 textures is not a good thing. I use the D3 stuff more than our own, and not just because we don't have much - because the D3 stuff is really good. There will no doubt be people that make high tech missions, or even total conversions (modern day stuff) of TDM.

 

I'll give up the 100Mb HD space compressed textures would have saved, gladly.

 

Edit: Saw your edit, one moment...

 

Edit2: Nope, the ones I have locally which haven't been added to CVS yet qualify for that, and they are as screwed up as the stock D3 ones.

 

There has to be a reason that D3 sets my value to 1. I keep suggesting it might be because I have a GF4 generation card. So D3 recognizes that and goes into failsafe backward support texture mode. And if that really is the cause, it's no longer a small portion of users. Hell, if half a gig of HD space was lost to not using compressed normalmaps I wouldn't care less, and it'll never be that high. We can still compress the diffuses.

[Gildoran]

It's not just HD space that's the issue, so much as memory... if we don't use compression, it might not be feasible to have lots of 1024x1024 normalmaps.

 

As for the GF4 card thing... the weird thing about that is you seem to be able to load both types of DDS files, just not at the same time.

 

It seems almost as though your copy of D3 isn't converting TGAs to DDSs when it loads the level...

[SneaksieDave]

Can we just proceed the way things have been prior to this, and seek a solution separately? I ran through D3 fine, the expansion fine, and so far TDM fine. If a solution is found, the textures can be swapped (especially since they'll be smaller). In the meantime, it's all just deadlocking texture and level design at a time when inspiration is hotter than it's been, and in the end it might be for no reason. I'd hate to see that steam lost for nothing. Yes there's a hurdle, but we're not getting over it. I'm suggesting we run around it and try to find a way over it separately.

[Gildoran]

Ok, will do... I'll starting copying over images from the highres repository and working on adding more of your textures.

[SneaksieDave]

Highres? I'm actually not sure whichever one is the real one. Isn't the highres stuff the really big bloated stuff? Like, 4Mb textures we panned a long while ago due to ridiculous sizes?

 

Also, should we separate this stuff out of this thread and give it its own? Hold on, let me find where we jumped off the tracks.

 

Edit: it looks like everything from here onward (or perhaps the next, but this is what started it):

http://forums.thedarkmod.com/index.php?s=&am...ost&p=73270

is safe to move into a separate thread. If anyone would be willing/able to do that, of course.

[New Horizon]

Highres? I'm actually not sure whichever one is the real one. Isn't the highres stuff the really big bloated stuff? Like, 4Mb textures we panned a long while ago due to ridiculous sizes?

 

Also, should we separate this stuff out of this thread and give it its own? Hold on, let me find where we jumped off the tracks.

 

Edit: it looks like everything from here onward:

http://forums.thedarkmod.com/index.php?s=&am...ost&p=73334

is safe to move into a separate thread. If anyone would be willing/able to do that, of course.

 

The hi-res stuff is the 1024 x 1024 3 to 4 mb textures that we set aside in favor of the 512 x 512 700kb compressed TGA's. The problem is that compressed TGA's aren't recognized by video cards, so they have to be reinflated back up to 1 to 1.5 megs...which amusingly is roughly the same size as a 1024 x 1024 DDS texture. DDS will give us higher resolution and better performance due to the use of mip maps for things that are far away from you...it will definately help large scenes. Consider using TGA's...you're in a large Hall...perhaps 60 feet wide. With TGA's the large wall on the other side of the room is going to be rendered using the full sized 1.5 meg texture, but with a mip mapped DDS, that wall might only be using an 80kb version of the same texutre and it should look just as good since it's so small.

 

That's why we really need to find a simple way to get this working. I know you use a lot of good doom 3 textures, but in a sense...it might be better to cut ourselves off from them completely and really force ourselves to get custom content in there. A lot of times, I think you're using Dark Mod content.

[SneaksieDave]

Feels like ping pong here. I agree, get it working by all means, I just think we need to move forward with what works now in the meantime. Stuff can always be converted later.

 

I don't think I could ever agree with cutting D3 out of the picture though - all that stuff is there, and is good stuff, and it's silly to suddenly not be able to use gigs worth of id-quality data, because we're insisting on using some format the game doesn't like to read for some users. And we never can say just how many potential players a move like this may put out. Especially when all we have to do is proceed as before. If it ain't broke...

[OrbWeaver]

There seems to be some confusion here between the disk space and the video ram issues. These are not the same. Supplying TGA textures will have a definite impact on disk space, but it will not impact memory because Doom 3 compresses the TGA images dynamically if the requisite cvars are set.

 

I don't see why we couldn't use TGA images in the master development repository, and then perhaps strip them out and go to exclusively DDS versions when the non-hires version of the mod is due for release.

[SneaksieDave]

Well that's good to know. I'm hoping that doesn't mean we have to go with the 4Mb normalmaps in the 'normal' CVS repository though, as the 1Mb versions have been working fine for ages. "No" problems. If we make that (the 1Mb) the standard, then whatever fenagling is desired - be it going higher or compressing - can be done in a separate repository.

 

Also, I'm a bit wary about putting DDS out there in the first place, into the minds of users (at least until/unless the compression problem is completely solved), because then we'll get the situation where FM X contains custom DDS textures, which breaks it for Y% of the population...

[Gildoran]

Supplying TGA textures will have a definite impact on disk space, but it will not impact memory because Doom 3 compresses the TGA images dynamically if the requisite cvars are set.

Another problem that I've noticed is that when D3 compresses normalmaps, they tend to have artifacts, whereas if we use precompressed RXGB normalmaps, the artifacts aren't there. (this is noticable at the edges between boards in wood/wall/boards_new_001)

 

Anyway, for now, I guess I'll put both a DDS normalmap and a TGA normalmap in the main repository, so those who can read RXGB won't have any artifacts, but those who can't will at least be able to load the textures. It sounds like you're suggesting we should use full 1024x1024 normalmaps instead of shrinking the hires normalmaps down to 512x512 when we put them in the darkmod repository?

 

@SneaksieDave: Can you load all TDM textures now? (assuming you set image_useNormalCompression back to 1)

[SneaksieDave]

Yes, as far as I can tell (aside from the new boards of course). I might have missed a step here. Is there anything else specific you wanted me to try?

[Gildoran]

I just uploaded TGA versions of all the DDS normalmaps, so you should hopefully be able to look at the new boards just fine. If you still can't after synching, please say so...

[SneaksieDave]

Yep, I can see the wood fine. I'll take texture specific stuff over to my texture thread.

 

Oh and whoever made brick006, omfg, that is one beautiful texture.

[Gildoran]

Thank you! I made it. It's a picture of the bricks used by my fireplace. The normalmap was generated with GIMP's normalmap plugin from a heightmap based off the photosource.

 

BTW, for any textures that are added, I've been adding comments to the material file mentioning their author.