How to open and edit the .dat file safely and successfully ? (Purposes: Font editing)

[Petike the Taffer]

Hello all.

 

One perpetual snag I keep hitting in my efforts to add or improve letters in existing fonts, for the purposes of translation, is the difficulty of accessing the .dat file that is apparently crucial for making the minor changes to the fonts stick, and allow these fonts to work properly with the added new letters.

 

Any advice on how to solve this issue, whether via external software or other solutions ?

 

I'm on Windows 7 32-bit, if that's in any way relevant to what I should get or how I should resolve this issue.

 

Thank you all for any advice.

 

 

[freyk]

Which dat-file?

Have you tried to change the filename extension from .dat to .zip?

 

@forumadmin

topic move to techsupport or art.

Edited February 2, 2018 by freyk

[Petike the Taffer]

Which dat-file?

Have you tried to change the filename extension from .dat to .zip?

 

@forumadmin

topic move to techsupport or art.

 

I think this should be moved to Art Assets. It's part of an i18n effort for one language. The fonts need adding to and editing for that particular language.

[stumpy]

usually to edit fonts you get the original true type font, use a free font editor that can load and save true type fonts, edit the font the way you want, then use the software, it used to be linked in the wiki. to change the modified true type font into the <filename>.dat and <filename>.dds files.

[Springheel]

Any changes to the fonts will need to preserve the work I did updating the menu fonts for 2.06.

 

 

 

[freyk]

There is also a tdm wiki-article:

http://wiki.thedarkmod.com/index.php?title=Font_Conversion_%26_Repair

[Petike the Taffer]

Any changes to the fonts will need to preserve the work I did updating the menu fonts for 2.06.

 

 

 

Springheel, it's actually very fortunate you've mentioned this !

 

Now I'm decided about whether I should put off the letter additions/repairs until after 2.06 is released. I should. No point in working with the old font, so I'll still have plenty of time to perfect that for 2.07. In the meantime, I'll practice the procedure, but I'll only edit the font once the updated version is standard.

 

As for the translation itself, 2.06 will get released with a stripped-down version, largelly without diacritics. (Only for the immediate future, I should be able to fix it by the time of 2.07. Hopefully...)

 

There is also a tdm wiki-article:

http://wiki.thedarkmod.com/index.php?title=Font_Conversion_%26_Repair

I know of that article and related articles, but it's a bit hard making sense of them.

 

usually to edit fonts you get the original true type font, use a free font editor that can load and save true type fonts, edit the font the way you want, then use the software, it used to be linked in the wiki. to change the modified true type font into the <filename>.dat and <filename>.dds files.

Just the other day, I was discussing good free font editors capable of this with Bikerdude and skina.

 

I still haven't found one, so if anyone has successfully used some software suited for the role, I'm all ears.

Edited February 3, 2018 by Petike the Taffer

[teh_saccade]

Petike - I was investigating the fonts last year or so and they are merely, as stumpy has said, .ttf that are renamed .dat and .dds.

Could never figure why there were also packaged as teh .dds, unless it was either for compression or for mapping (not level mapping, like... normals / specular or whatever). The .dds don't appear used by the game, but idk jack.

I can't remember if it was you who was doing the Russian translation - from the sounds of it probably not - but there are not many fonts that will jibe with TDM that support Cyrillic characters:
http://www.wazu.jp/gallery/Fonts_CyrS.html

A few stick out (eg, Arbat) as in-line with the game, but most are... not that great. It's a problem. Mostly, for subtitles, some unicode sans-serif / arial is good enough.
Most are designed, bespoke. Check out vk.com typographic groups, perhaps.

As for diacritic marks, the choice is even more limited for universal support utf-8:
https://collab.its.virginia.edu/wiki/toolbox/Windows%20Unicode%20Diacritic%20Fonts.html
You will be able to find other OS from that link.

In wanting to make a series of cyphered messages, it required simple problem solving for the player only - which might've been an annoying puzzle - so I wanted to make a fake language or series of glyphs representative of English letters that the player would have to decode using a codex, found as part of an objective.

I use a piece of kit called "Glyphs" that is decent for digital type-foundry - if you have inkscape or illustrator, it would be possible to convert by scanning even handwriting (or old books) into vector format and import these vectors into a font-maker.

Glyphs is a few hundred quid, though...

A good, free digital type-foundry - that also allows for this process of scanning rasterised (bitmap) lettering to vector, or importing inkscape/illustrator vector glyphs/letters (I believe) to .ttf (to .dat) for use in the game might be:
https://birdfont.org/

It has tutorials and I've heard good things.

Idk if it can import and rip .ttf's, but... what I do is use my laptop monitor as a lightbox and make the glyph/letter superlarge on the screen and trace it before colouring it black with a sharpie with nice, clean edges (or not...).
Then I scan it and trace it with beziers (good practice to snap them at 45 degrees) or, occasionally - illustrator's autotrace feature sometimes does a decent job. I do not know if Inkscape (free) has this function. I've not used it for a long time and I'm never going back.

However, Inkscape is capable of importing .ttf fonts in order to expand to vector, to copy into a free font editor (I think, can trace a bitmap, at least)..:
http://wiki.inkscape.org/wiki/index.php/Installing_fonts <--- not a problem
https://inkscape.org/en/doc/tutorials/advanced/tutorial-advanced.en.html

One thing to watch out for is the tracking / kerning / leading (space between letters, letters in words and space between lines of text) - there is a section on how to achieve this.

There's a reason why people value decent typefaces at such a high price. It's a fair bit of work.
Personally, I like to collect old wooden and metal block press lettering and press the glyphs and filigree and scan that to use... like a printing press. Placing the letters one at a time for more control. Plus people don't recognise the font. I even use gallium metal to cast my own, if required.

It might be worthwhile taking a look through the myriad font sites that offer free for non-commercial use fonts, such as:
http://www.fontspace.com/category/diacritics

TTF RenderText Blended fonts will support ASCII characters (which is why some of the free fonts that say "diacritic" might not show up with the correct markings), but if they are unicode then it is likely it may likely appear as [] in whatever.

If there can be no suitable font found - I'd say that using a free piece of kit such as Inkscape to draw (or trace from bitmap) the lettering, to export / copy-paste as vector into Birdie, to set up the font - then export as .ttf to rename as .dat would be the free-route to achieving your goal.

Using this work-flow, there's no reason as to why the existing fonts might not be altered to include the extra marks you require for your translations.
Just gotta add a few dots and some squiggles in a similarly degraded look, or remaster the entire waterfall for consistency. Or transliterate.

Steep learning curve, lots of swearing in frustration, hard to understand... much like the vast majority of TDM/DR wiki;)

[teh_saccade]

I might have that bit about ascii and unicode backwards... i'm not sure. It's definitely one or the other.

 

// i'd say google it but it's probably irrelevant.

Edited February 11, 2018 by teh_saccade

[Petike the Taffer]

I might have that bit about ascii and unicode backwards... i'm not sure. It's definitely one or the other.

 

// i'd say google it but it's probably irrelevant.

 

Thing is, my preferred method would be editing the .dds of the individual fonts (only where needed) and then saving those (or, if needed, converting them to the needed format afterward). I am actually capable of opening .dds files in Paint.net.

[teh_saccade]

Are you able to actually open the .dds files in paint.net..? It's not a bitmap - it's a hack-wrap.

 

System.FormatException: File does not appear to be a DDS image



   at PaintDotNet.Data.Dds.DdsFile.Load(Stream input) in D:\src\pdn\src\PaintDotNet\Data\Dds\DdsFile.cs:line 469



   at PaintDotNet.Data.Dds.DdsFileType.OnLoad(Stream input) in D:\src\pdn\src\PaintDotNet\Data\Dds\DdsFileType.cs:line 225



   at PaintDotNet.FileType.Load(Stream input) in D:\src\pdn\src\Data\FileType.cs:line 496



   at PaintDotNet.Functional.Func.Eval[T1,TRet](Func`2 f, T1 arg1) in D:\src\pdn\src\Base\Functional\Func.cs:line 158

 

It won't parse for me - the .ttf renamed as .dds won't parse in ANY image editor, because it is not a bitmap - it's simply a .ttf renamed so the game engine can snag it using arcane or pagan magic of some kind.

TL;DR - use a .ttf editor and, if it won't import and/or you can't draw - trace it, either over (insert glyph under and trace/alter) or under (print the glyph onto thin paper-stock and stick it on your screen and trace/alter).

Even though .dds is a bitmap format - it appears that it's merely used as a hack-wrap for the font in this instance so that TDM can understand it. Same as the .dat (which is the convention used in pretty much every TDM FM).

As an alternative, if the idea of doing things the easy way is intimidating - it might be worth checking out the old Doom3 font.dat editor:
https://github.com/Zbyl/BFGFontTool
which would make the process only a little more complicated.

Perhaps you might find Fontforge easier to use, as it can add accents to existing fonts:

Here's an alternative workflow that might prove easier for you (bring the pain!).

1. Copy the font.dat you want to edit to your desktop.
2. Rename the font.dat to font.ttf.
3. Install the font by double-clicking the .ttf.
4. Open Inkscape and add the character from the font you wish to edit.
5. Expand the font [object] to path [vector] (ctrl+shift+C) - a tutorial on the process is here: http://studios.clockworkmagpie.com/content/how-make-outlined-text-inkscape <--- only need to do the expand - but, the more you know... // not g, sorry.
6. Export the vector as .svg, ready to import into Fontforge.
6a. You will need to do this for each and every character you want to alter, individually; I'm not sure there's a way to batch this using these programmes.
7. Import vector from inkscape into Fontforge - a tutorial on the process is here: https://fontforge.github.io/importexample.html
7a. If you're up for a challenge, or for some reason the font doesn't correctly appear, printscreen the character and trace it, manually or with the... "tool" provided: https://inkscape.org/en/doc/tutorials/tracing/tutorial-tracing.en.html
8. Use the imported vector [character].svg in Fontforge and add the diacritic you need: http://designwithfontforge.com/en-US/Diacritics_and_Accents.html <--- grab Fontforge from git from this link.
9. Rinse and repeat - I can't remember if there's a way to add (alternative) characters to a font using Fontforge, or if you'll have to do every character from the original .ttf.
9a. Don't forget to take into account the technical stuff for character spacing, etc..: http://designwithfontforge.com/en-US/Spacing_Metrics_and_Kerning.html
10. Export the new font you've created as _sl_24.ttf. Hopefully it'll install so you check it out (either in text editor or as html in browser). Rename to .dat and test it in a readable.
11. Maybe the font can be patched to include the additional characters, but do not believe it would be necessary as you've essentially created a well and accurately-crafted version of the existing typeface, no spacing errors. There's a wiki article about that somewhere.
12. Job's done, pat yourself on the back.
13. For good luck - take a 13.

That's it - it's a bit of a learning curve, will take time, but it's the way I'd do it.

From this point on, I'm pretty useless to you wrt to fonts in TDM.
That's the extent of my typographic skill and knowledge and problem solving ability.

You're not simply "cleaning" up a typeface using a click of a button and a few coding commands - what you're doing is editing the font, like a real typesetter. As how it was made, originally.

In fact, you're creating a new version of the font.
Kudos.

----

Afaik - the .dds are the same as the .dat - I could tell no difference, which is why the speculation that it was for compression or placeholder/alternative/for additional maps for the font.

If you don't like to use Bird - I tried it out and got along with it fine, although it's a little laggy and kind of assumes familiarity from the start - to edit the .ttf and follow the instructions so the .ttf is recognised by the game by simply renaming the resulting font as .dat (conventional) or .dds (apparently not used often, if at all)...

Awkwardly - perhaps you can use the method similar to games such as old M&B custom fonts, but for doom:

http://www.angelcode.com/products/bmfont/ <--- bitmap font editor, edit your font, save as .ttf / .dds / .xml (for formatting), etc...
http://www.angelcode.com/products/pupp/ <--- specifically for .dds (even though the .dds is a renamed .ttf, it might work).
https://github.com/Zbyl/BFGFontTool/ <--- doom3 font.dat [f/m]aker (requires angel bmfont, never tried it)
https://www.aconvert.com/image/ttf-to-dds/ <--- online converter, in case of jinks (poss. redundant due to .dds being merely renamed .ttf and not really .dds - also doesn't convert back to .ttf, heh - perhaps a rename will work to .dat, idk... doubtful, as the .dds is not a bitmap, but a renamed .ttf - not sure i've mentioned that...).

Considering all the stuff is gone that Swyter made to do it for us in a browser, you might need some help understanding how to mess with .xml output in order to adjust the font:
http://helpsharepointvision.nevron.com/XML_Formatted_Texts.html

But... even if you could simply convert the .dds bitmat to .dat with a file extension change... that still leaves how to incorporate all the adjustments made using the xml in order for it to correctly display...
Plus - it's not technically a bitmap - it's a .ttf FONT that has been RENAMED and NOT CONVERTED TO A BITMAP as this appears to happen somewhere in the game engine and I'm too dumb to understand how this works.
Soooo... for that part it makes sense to be working backwards and setting up the .ttf with .xml BEFORE converting to .dds... which would be a lot of hassle and frustration.
But... it's not necessary to change to .dds, since:

a. you've already edited the font using a bitmap editor that's free and easy
b. there's birdie, which does everything from the .ttf that's wrapped up as .dat/.dds (if you can't draw, simply print the character on a piece of paper and tape it over your screen and trace it underneath - or, copy paste the character from the original glyph and add the marks you need).
c. the .dds isn't a bitmap, it's a hack-wrapped .ttf font file.
d. the missions appear to mostly use .dat not .dds... because that's the way the wiki says; it's the convention that appears everyone uses.

As for getting every FM to use .dds rather than the .dat - that's likely an overhaul of every mission to use .dds fonts instead of .dat, idk.
I don't think anyone particularly cares much about that side of things right now - it's all about models and squares and graphics and optimisation and breaking the lightspeed barrier.

 

I've not spotted any use of .dds files in any TDM FM as the convention appears to be to use the .dat.


I'd prefer a lot of things to be different about my workflow, but even though I can hammer in a nail with a wrench - it doesn't mean I can tighten a bolt with a hammer.

I've no idea why there are two standards for TDM fonts, so perhaps pretending that the .dds isn't there might help..?

Here - have some sad, soothing music to listen to while sitting in the corner and sobbing quietly about how it feels to live in a tiny fraction of my world:

All I figured about the fonts is that they're .ttf that are renamed to .dat and they work if you do that.
Therefore, instead of working with the extension in a bitmap editor (which doesn't work, as it's a renamed .ttf and not a bitmap, in case I didn't mention that yet) - it makes sense to make alterations to the source, as there may be more to it opening up what may or may not be able to be converted to bitmap and chucking a few dots over it before flicking that bitmap back to a .ttf to format (either using xml or a free font editor...) and rename to .dat or .dds.

_{Which'd be a fucking nightmare.}

All the fonts are .ttf.
They're simply renamed so the game engine can use them.

That's all there is to it.

If it was a case of simply editing a bitmap image - I wouldn't have taken the time and gone to such extent to explain the process of how I would do all this if I had to...


Sorry, mate - sometimes you gotta just "grab the uplay and bite the pillow".

I can't be more concise than this.

If you need any help understanding any of this - ping and I'll lend what assist I can, depending available; plate's full.

GLHF

Edited February 13, 2018 by teh_saccade

[teh_saccade]

forgot. The characters you'll need, other than your own:

 

!"£$%^&*()_+.-./#1234567890;:<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]`abcdefghijklmnopqrstuvwxyz{|}~

 

(maybe not currency, since it's usually "gold/silver/copper pieces")

Valentine's Rose: @~8~~~
Penis: 8==D
Breasts: ( . Y . )
etc...

(super important for readables)

For .dds font, with shaders and specular maps and stuff - usually it requires something like this:

http://www.rastertek.com/dx10tut12.html

So - not knowing how TDM handles fonts, I'm starting to think that the .dds is included as alternative hack-wrap for game engine and not for including such things for fonts to display in-game. (maybe i'm wrong, but idk about that stuff, really - esp. not with TDM)

Thanks for helping me kinda figure that one out... can scratch that off the list of things that keep me awake at night:P

Edited February 13, 2018 by teh_saccade