Jump to content
The Dark Mod Forums

Ported font code from Doom3 BFG Edition


Zbyl

Recommended Posts

Hi guys,

 

I haven't seen anyone working on this, so I've ported font code from Doom 3 BFG edition to The Dark Mod. This allows to create fonts that support full unicode range.

I'm porting LangDict now, so all texts will be in unicode and no hacks will be necessary - no code pages, conversions, replacements, separate font files, problems after language change etc.

 

There's one issue: the BFG fonts don't have three sizes (small, medium, large). There is only one font size, and this font is scaled if necessary.

This creates an unfortunate side effect: font scales in GUI that were tuned for old fonts will be wrong now.

In the Doom3 BFG they do a little hack: they load old fonts too, but use them only to emulate old font measuring code. This is not only ugly, but also a maintenance nightmare. For TDM I would propose to drop it and just fix the GUI to use proper font scales.

 

I'd like to know what you guys think about this. (To make it clear: using old and new fonts together is not possible.)

 

Some technical stuff: I've used some new helper files from BFG. They are not strictly necessary, but I didn't see any reason not to use them if it made my work easier. Those are for example: sys_assert.h, sys_types.h, new collections/List.h any maybe something else.

The porting itself was really straightforward - copy/paste mostly.

I've also made a little utility program that converts fonts created by a freeware BMFont (http://www.angelcode.com/products/bmfont/‎) into BFG fonts.

 

Another thing connected to fonts and localization that I'd like to discuss are the toUpper, toLower etc. functions. Those cannot be reliably used for GUI. Think about toLower() for german language: sometimes "SS" have to become "ss", and sometimes "scharf ss". It should be translators job to prepare text in proper case and form. I strongly advise against any conversions like that in GUI output.

The toUpper() and toLower() functions in TDM right now don't work properly for diacritics (like ąęćżźńł, scharf ss) - and it makes GUI look super ugly.

However: providing implementation that will work decently is quite simple and I've written most of the code for it.

The question is: should I finish it, or should TDM just drop toUpper from GUI code?

 

(BTW.: I know that you won't even consider integrating any of this stuff before 2.0 is out.)

Edited by Zbyl
Link to comment
Share on other sites

With all the manual painting Tels put in editing the existing raster fonts to add previously unsupported characters, I imagine it would require a lot of effort to use custom fonts in an FM. If the process can be made easier, code freeze aside, I don't see why anyone would be opposed to it.

Link to comment
Share on other sites

@Zbyl: What tool do you use to generate BFG fonts?

 

BFG's fonts improvements were meant to improve Doom 3 fonts - make them of higher resolution and allow for extra characters, that's all. They were not a replacement for original font system. That's why BFG's fonts need old font .dat files to map new characters over them.

Link to comment
Share on other sites

@Zbyl: What tool do you use to generate BFG fonts?

Like I said: I use BMFont and my custom converter from BMFont to BFG fonts.

BFG's fonts improvements were meant to improve Doom 3 fonts - make them of higher resolution and allow for extra characters, that's all. They were not a replacement for original font system. That's why BFG's fonts need old font .dat files to map new characters over them.

They do replace the original font system (they need original fonts only to know what their maxCharWidth/maxCharHeight is to know how to scale new fonts into old GUI). This is in the code - no guesses here.

 

What's the problem this port solves? I.e. why do we want it?

There are many code pages to support now - language files need to be converted into custom encodings.

Russian language is hacked in quite ugly.

There is no way to display i.e. language list in the GUI properly: i.e. russian in russian, polish in polish, spanish in spanish - no one font supports all the necessary characters.

Separate font files - inconvenient.

Need to reload fonts after language change - inconvenient.

Adding new languages is tricky.

There's no way TDM can support Japaneese, Chineese, Korean etc. right now. It would be a breeze with BFG's fonts.

Basically the I18N support is a big hack right now.

 

With all the manual painting Tels put in editing the existing raster fonts to add previously unsupported characters...

 

I'd like to get a look at this. Is this code somewhere in the public SVN?

The BMFont I used doesn't support this - but I used BMFont just because it was the easiest way I could generate a font for my testing. It would probably take little effort to generate BFG fonts using the content pipeline you already use.

Even better: if breaking backwards compatibility in any way is not an option then BFG code can be modified to work just like the old system, but without the old D3 problems and with unicode support.

But breaking backwards compatibility is ok sometimes. I just wonder what course of action would be best.

 

i use custom fonts in some of my maps it sounds like this change would break the fonts i use.

It would, but there are ways to solve it. It could be done the hacky way (just porting current behaviour).

But since the full D3 code is open now - it could be done the proper way: i.e. support changing font to a custom one mid-text, like the support for changing colors is done.

 

About the broken toUpper()/toLower(): any thoughts on that?

(I've described some of the problems here in http://bugs.thedarkmod.com/view.php?id=3321)

Edited by Zbyl
Link to comment
Share on other sites

But breaking backwards compatibility is ok sometimes.

 

Seems like this would break a lot of maps and require a lot of adjustments to the guis and translation system. Having spent the better part of a year fixing compatibility issues caused by going standalone, the benefits would have to be pretty impressive to make it worth the hassle.

Link to comment
Share on other sites

I think the benefits of this code are impressive, even if not evident at first, so I'll keep working on this.

It would definitely help me with compatibility to know what tools you use for generating fonts. Is it ExportFontToDoom3 exclusively?

 

I also think I misunderstood the "manual painting Tels put in editing the existing raster fonts". I thought he wrote some tool to support adding new glyphs to existing fonts, but now I re-read it it sounds like you were talking about actual editing of font textures. It definitely would not be a wasted work. The only thing that is needed is: some tool to support adding new glyphs to existing fonts... I'll look into this too.

  • Like 2
Link to comment
Share on other sites

I think you have a far better understanding of the system than most of us, so it might be hard for us to understand the benefits. So this will also improve the look of the fonts as well as under the hood functionality? If you think it's worth it, i say go for it and we can look at the pros / cons after 2.0 has been released.

 

Have you looked at any of the performance side changes in Doom 3 BFG? Things like dual core enhancements and threading? Those things would probably be a nice performance boost for us later on down the road.

Link to comment
Share on other sites

Like I said: I use BMFont and my custom converter from BMFont to BFG fonts.

 

BMFont's page shows 404. As far as I remember, BMFont generated glyph atlas from TTF fonts. How do you generate .dat file for BFG? Can you upload the final BFG font you generated, using BMFont and your secret tool, so I could look at TGA atlas and .dat file, please?

 

They do replace the original font system (they need original fonts only to know what their maxCharWidth/maxCharHeight is to know how to scale new fonts into old GUI). This is in the code - no guesses here.

 

Right, they didn't replace it. You can still use old fonts with BFG, or just new font. However, it's be off. Replacing means you can use new font without screwing GUIs. They added new font system without removing old one.

 

EDIT: http://www.angelcode.com/products/bmfont/ works, which doesn't solve the issue of generating proper .dat files

Edited by motorsep
Link to comment
Share on other sites

Have you looked at any of the performance side changes in Doom 3 BFG? Things like dual core enhancements and threading? Those things would probably be a nice performance boost for us later on down the road.

 

This and all of my this. This contribute of yours might seems like a nice addition, but what we really need atm is a bad dude enough to tackle this one with wrestler's arms.

Link to comment
Share on other sites

Can you upload the final BFG font you generated, using BMFont and your secret tool, so I could look at TGA atlas and .dat file, please?

My test Gothic font is here: https://drive.google...N1U&usp=sharing

BFG .dat is of course in a different format than D3 .dat. The TGA is generated by BMFont - I didn't make any chages to it.

BFG fonts are different for example in that they use only one big texture per font, not many like in D3. This is not a problem these days when big textures are universaly supported.

 

As for the performace improvements: I don't know a lot about this. Only stuff I've read in BFG code review and some articles by Jean Paul van Waveren and Carmack. But I'll take a look if I'll find the time.

Link to comment
Share on other sites

My test Gothic font is here: https://drive.google...N1U&usp=sharing

BFG .dat is of course in a different format than D3 .dat. The TGA is generated by BMFont - I didn't make any chages to it.

BFG fonts are different for example in that they use only one big texture per font, not many like in D3. This is not a problem these days when big textures are universaly supported.

 

Ahh, I see. So font atlas is of the power of two, while original BFG atlases are non-power of two. BFG's glyphs are packed very tightly, rotated/mirrored if needed to fit more characters in one atlas. I suppose it doesn't really matter for regular fonts, but for Japanese/Korean/Chinese fonts just square atlas would be huge :/

 

Have you tested if your font works with BFG (converting ArialNarrow into BFG's font and replacing the original ArialNarrow that comes with BFG) ? If not, and you don't really know how, I can test it if you can convert ArialNarrow.

 

Do you convert glyphs descriptor generated by BMFont into .dat file? If so, can you please share the code of your converter?

Link to comment
Share on other sites

Do you convert glyphs descriptor generated by BMFont into .dat file? If so, can you please share the code of your converter?

Yep. I convert the text version of the descriptor into a BFG .dat. Probably converting XML descriptor would be easier, but I just didn't thought of it.

Sure I'll share the code - but at the moment it's rather messy. And in C#. I'll clean it up a tiny bit first.

 

As for testing my font in BFG - sadly I can't do that, since I don't own BFG... I only have the code to play with.

Link to comment
Share on other sites

Yep. I convert the text version of the descriptor into a BFG .dat. Probably converting XML descriptor would be easier, but I just didn't thought of it.

Sure I'll share the code - but at the moment it's rather messy. And in C#. I'll clean it up a tiny bit first.

 

As for testing my font in BFG - sadly I can't do that, since I don't own BFG... I only have the code to play with.

 

Thanks! Ahh, C#.. I hope we can convert it into C++ :)

 

If you can send me ArialNarrow converted with your method, I can test it here (.tga and .dat is what I need).

Link to comment
Share on other sites

As for the performace improvements: I don't know a lot about this. Only stuff I've read in BFG code review and some articles by Jean Paul van Waveren and Carmack. But I'll take a look if I'll find the time.

 

That would be greatly appreciated, as TDM currently suffers from some serious performance issues on older pcs, and even some choppy frames on modern pcs when playing larger maps.

Link to comment
Share on other sites

I've created an ArialNarrow font that should work. I'm not sure if the scale will be ok though. https://drive.google...ejg&usp=sharing

I've attached BMFont's config and .fnt that I used to create this font too.

 

Ok, tested. Here are the comparison screenshots: http://www.steel-storm.com/files/d3bfg_ArialNarrow_font_comparison.zip

 

Generally speaking, it's a very good start! However, as you can see, there are issues. Font size doesn't match up, somehow there are white boxes around glyphs (_09.tga; terminal GUI on Mars City 1 at the very beginning right off the drop ship) and it's totally screwed up in PDA.

Link to comment
Share on other sites

Thanks for this test motorsep.

The white boxes appear because I apparently gave too small spacing in BMFont's generation options. I should have bumped the output resolution too.

I need to figure of what settings to use for scaling fonts properly. In D3 standard fonts are of size 48pt, but with what DPI? Maybe 120?

And why the PDA is messed up I have no idea... yet.

Link to comment
Share on other sites

The reason why my font display incorrectly in PDA is this code in BFG's SWF text renderer:

float linespacing = fontInfo->GetAscender( 1.15f * glyphScale );

My ascenders are 0 now, since they are not used anywhere in text renderer.

(This code is used only in the SWF renderer of the new Doom GUI).

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recent Status Updates

    • Ansome

      Finally got my PC back from the shop after my SSD got corrupted a week ago and damaged my motherboard. Scary stuff, but thank goodness it happened right after two months of FM development instead of wiping all my work before I could release it. New SSD, repaired Motherboard and BIOS, and we're ready to start working on my second FM with some added version control in the cloud just to be safe!
      · 0 replies
    • Petike the Taffer  »  DeTeEff

      I've updated the articles for your FMs and your author category at the wiki. Your newer nickname (DeTeEff) now comes first, and the one in parentheses is your older nickname (Fieldmedic). Just to avoid confusing people who played your FMs years ago and remember your older nickname. I've added a wiki article for your latest FM, Who Watches the Watcher?, as part of my current updating efforts. Unless I overlooked something, you have five different FMs so far.
      · 0 replies
    • Petike the Taffer

      I've finally managed to log in to The Dark Mod Wiki. I'm back in the saddle and before the holidays start in full, I'll be adding a few new FM articles and doing other updates. Written in Stone is already done.
      · 4 replies
    • nbohr1more

      TDM 15th Anniversary Contest is now active! Please declare your participation: https://forums.thedarkmod.com/index.php?/topic/22413-the-dark-mod-15th-anniversary-contest-entry-thread/
       
      · 0 replies
    • JackFarmer

      @TheUnbeholden
      You cannot receive PMs. Could you please be so kind and check your mailbox if it is full (or maybe you switched off the function)?
      · 1 reply
×
×
  • Create New...