Jump to content
The Dark Mod Forums

Convert normal maps to RGTC?


duzenko
 Share

Recommended Posts

3 hours ago, STiFU said:

Do you have a converter for RGTC @duzenko? I'd like to conduct an experiment so see how loading times are affected. I assume their is no code support needed to load RGTC normals at this point?

Good question that I can't answer fully ATM. D3 offers two automatic conversion options:

  • via the r_writeCompress_something cvar that is supposed to let the GL driver do the conversion and then simply bit copy the result from the VRAM. You'd think that driver's on-the-fly compression is fast but image quality might suffer.
  • using an external command-line tool. Supposedly this is the recommended option. We still need to decide which 3rd party compressor we want to use and how to wire it to TDM (command line params, etc)

Feel free to try either of these but keep in mind that texture loading is only a fraction of total map loading time.

I'd like to get the RGTC conversion completed but afraid to touch it before 2.08 is out

Link to comment
Share on other sites

40 minutes ago, duzenko said:

Feel free to try either of these but keep in mind that texture loading is only a fraction of total map loading time.

How can you tell? Did you profile map loading? If so, I'd be very interested to see the results!!

Quote

I'd like to get the RGTC conversion completed but afraid to touch it before 2.08 is out

Yes, this is definitely a 2.09 task.

Link to comment
Share on other sites

20 minutes ago, STiFU said:

How can you tell? Did you profile map loading? If so, I'd be very interested to see the results!!

Yes, this is definitely a 2.09 task.

Unfortunately VS2019 profiling does not work for me

But image_preload 0 will do the job - try it.

Link to comment
Share on other sites

 

17 hours ago, STiFU said:

Do you have a converter for RGTC @duzenko? I'd like to conduct an experiment so see how loading times are affected. I assume their is no code support needed to load RGTC normals at this point?

If you have time to have fun, you can try to implement your own in something like Python. All those texture compression formats are so simple compared to PNG and JPG, that writing a basic converter is a one-day problem ūüėé

13 hours ago, duzenko said:

Unfortunately VS2019 profiling does not work for me

I'm afraid CPU profiling is not the right way to conduct experiments in this case. There are many things involved which you won't be able to properly measure with profiler: disk accesses, OpenGL driver work, GPU transfers, etc. I suggest doing more complicated measurements with temporarily disabling some parts in the code. I guess most of the time should be taken by texture loading, models loading, and sound loading (also covered in zip compression unless you are on SVN).

Link to comment
Share on other sites

2 hours ago, stgatilov said:

 I guess most of the time should be taken by texture loading, models loading, and sound loading (also covered in zip compression unless you are on SVN).

.map/.proc/.cm files can take a while to load, as well as AAS parsing

Then, there are also scripts, def's, materials

Link to comment
Share on other sites

On 3/26/2020 at 8:01 PM, duzenko said:

Good question that I can't answer fully ATM. D3 offers two automatic conversion options:

  • via the r_writeCompress_something cvar that is supposed to let the GL driver do the conversion and then simply bit copy the result from the VRAM. You'd think that driver's on-the-fly compression is fast but image quality might suffer.
  • using an external command-line tool. Supposedly this is the recommended option. We still need to decide which 3rd party compressor we want to use and how to wire it to TDM (command line params, etc)

Feel free to try either of these but keep in mind that texture loading is only a fraction of total map loading time.

I'd like to get the RGTC conversion completed but afraid to touch it before 2.08 is out

What about ATI/AMD compressionator? It was the tool that idsoftware used to compress normal maps for Doom 3 and splash damage for quake wars, you can find the old version used in Doom 3 here.

Or perhaps ODCompress from Overdose?

Link to comment
Share on other sites

3 hours ago, duzenko said:

Does either of them support RGTC?

Don't know a thing about texture compression but latest compressionator "Can compress to a wide range of compression formats including ASTC, ATC, ATInN, BCn, ETCn, DXTn and swizzled DXTn formats, and supports conversion of textures between 8bit fixed, 16bit fixed and 32bit float formats."

 

Taken from this link:   "ASTC provides a single-channel and two-channel version, equivalent to RGTC's formats."

  

Edited by HMart
  • Like 1
Link to comment
Share on other sites

As I recall, RGTC is just another name for Microsoft's BC4 or 5

This seems to confirm it:

RGTC1_RED = BC4U, RGTC1_SIGNED_RED = BC4S, RGTC2_RG = BC5U, RGTC2_SIGNED_RG = BC5S

 

Please visit TDM's IndieDB site and help promote the mod:

 

http://www.indiedb.com/mods/the-dark-mod

 

(Yeah, shameless promotion... but traffic is traffic folks...)

Link to comment
Share on other sites

6 hours ago, HMart said:

Don't know a thing about texture compression but latest compressionator "Can compress to a wide range of compression formats including ASTC, ATC, ATInN, BCn, ETCn, DXTn and swizzled DXTn formats, and supports conversion of textures between 8bit fixed, 16bit fixed and 32bit float formats."

 

Taken from this link:   "ASTC provides a single-channel and two-channel version, equivalent to RGTC's formats."

  

Can't see any reference of a compression tool on that link

Link to comment
Share on other sites

4 hours ago, duzenko said:

Can't see any reference of a compression tool on that link

Nor was that my intention I was just showing that I've read that ASTC is equivalent to RGTC but I don't know if is really true or not. 

Ha now I know what you meant by that, the first quote in dark gray from my reply above was from the latest AMD compressonator page, the link to it is on my reply above with the links for the tools, the other quote in white, is from another link, the "taken from this link" was referring to the white quote only.    

Edited by HMart
Link to comment
Share on other sites

  • 7 months later...
  • 7 months later...

I haven't been keeping up with the normal map compression changes, but I am confused about the current correct way to produce a DDS normal map.

Exporting from GIMP, both DXT1 and DXT5 work fine, but have the blocky artifacts I associate with DXT-compressed normal maps. "RXGB" produces a normal map which is clearly missing one dimension, so I guess the DXT5 swizzling has been disabled.

From the link @nbohr1more posted it seems like "BC5 / ATI2 (3dc)" (which exists in the GIMP plugin) ought to be equivalent to RGTC, but this only gave me garbage results. I don't know if this is the wrong format to choose or if it is a limitation of the GIMP plugin itself.

Link to comment
Share on other sites

  • 4 weeks later...

dds is a standard format nowadays, and since those pesky patents on it expired it should be safe to use in FOSS projects. Gimp seems to work fine with it (importing and exporting) so it should be okay for artists editing the graphics. Especially if it's going to improve those huge loading times and RAM usage, it does feel worth considering to me.

  • Haha 1
Link to comment
Share on other sites

We're already using DDS format, and have been doing so since the initial release of the mod.

The recent comments in this thread are about using RGTC compression within DDS for normal maps, which should give better quality than regular DXT1 but is not necessarily as widely supported by editing tools.

Link to comment
Share on other sites

6 hours ago, OrbWeaver said:

We're already using DDS format, and have been doing so since the initial release of the mod.

The recent comments in this thread are about using RGTC compression within DDS for normal maps, which should give better quality than regular DXT1 but is not necessarily as widely supported by editing tools.

I see. Are Gimp / Krita / Photoshop able to load and save that, or at least plan on being able to? If yes then I'd say go for it.

Gimp is the image editing software I actively use so I gave it a try. I can see there's no export option for RGTC which does raise a concern for this plan.

Link to comment
Share on other sites

1 hour ago, MirceaKitsune said:

I see. Are Gimp / Krita / Photoshop able to load and save that, or at least plan on being able to? If yes then I'd say go for it.

Gimp is the image editing software I actively use so I gave it a try. I can see there's no export option for RGTC which does raise a concern for this plan.

You don't _need_ to convert your custom textures to RGTC

This thread is about the core assets only

Link to comment
Share on other sites

16 minutes ago, duzenko said:

You don't _need_ to convert your custom textures to RGTC

This thread is about the core assets only

I know. I was thinking whether this format allows them to be editable and accessible by mainstream editing tools, in case someone wants to look at them or make their own versions for development purposes. I'd think that should be possible, so people who need access to them won't need to struggle with conversion tools.

Link to comment
Share on other sites

5 hours ago, MirceaKitsune said:

I see. Are Gimp / Krita / Photoshop able to load and save that, or at least plan on being able to? If yes then I'd say go for it.

Gimp is the image editing software I actively use so I gave it a try. I can see there's no export option for RGTC which does raise a concern for this plan.

Supposedly yes (I believe it's equivalent to "3DC+" in the GIMP plugin), however as I mentioned I have not been able to get this to work at all with the latest build. Only plain DXT-compressed normal maps render correctly.

Link to comment
Share on other sites

2 minutes ago, OrbWeaver said:

Supposedly yes (I believe it's equivalent to "3DC+" in the GIMP plugin), however as I mentioned I have not been able to get this to work at all with the latest build. Only plain DXT-compressed normal maps render correctly.

How did you test this?

Link to comment
Share on other sites

Just now, duzenko said:

How did you test this?

I took a normal map which was working, then saved it using the GIMP plugin in 3DC+ format (I also tried using the Compressonator with various formats which looked like RGTC, e.g. "ATI2N"). After this, the normal map would either show only a single channel (e.g. horizontal bumps are visible but vertical bumps are missing), or no bumps at all; I don't recall exactly which.

Link to comment
Share on other sites

11 hours ago, OrbWeaver said:

I took a normal map which was working, then saved it using the GIMP plugin in 3DC+ format (I also tried using the Compressonator with various formats which looked like RGTC, e.g. "ATI2N"). After this, the normal map would either show only a single channel (e.g. horizontal bumps are visible but vertical bumps are missing), or no bumps at all; I don't recall exactly which.

Could you upload a same texture for me too look at?

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

 Share


  • Recent Status Updates

    • Gilkar

      When I was a young man my father was so ignorant I could hardly stand to have him around. As I grew older I was amazed at how much the old man had learned in such a short time.
      · 0 replies
    • jaxa

      RTX 3090 Super, RTX 3070 Ti 16 GB, RTX 2060 12 GB
      https://wccftech.com/nvidia-launching-rtx-3090-super-rtx-3070-ti-16gb-and-rtx-2060-12gb-by-january-2022/
      · 0 replies
    • duzenko

      CPU benchmark time - compiling DarkRadiant (2nd run)
      i5 8600K 6C/6T@4.4GHz DDR4 2x2133MHz 9MB cache
      Parallel builds: 1. 3:57 Parallel builds: 6 (default). 2:28 r5 1600AF 6C/12T@3.3GHz DDR4 1x2666MHz 16 MB cache, temp folder on HDD
      Parallel builds: 1. 5:05 Parallel builds: 4. 2:47 Parallel builds: 6. 2:55 Parallel builds: 12 (default). 2:57
      · 6 replies
    • nbohr1more

      Status updates are back so it is also a good time to return to contests!
      https://forums.thedarkmod.com/index.php?/topic/21095-christmas-connections-contest-2021
       
      · 0 replies
    • freyk

      Having seen the new scifi stuff from bladeghost/scythwraith, want to continue the cyberpunk project. Only one problem, can't map.
      · 1 reply
×
×
  • Create New...