FreeBSD native port

[Partmedia]

Dear community,

I've been working on porting The Dark Mod natively to FreeBSD. I'm happy to report that at this stage, I have a working thedarkmod.x64 and tdm_installer executables on FreeBSD 13 amd64 (x86-64) with SVN revision 9889 and 2.10 game files. I've been able to play through both default missions as well as several contributed missions.

There were minimal changes required to get a working port, including:

1. Fixing the build with Clang, FreeBSD's system compiler (committed in 9889 by Stephan)
2. Fixing up header files and #ifdef's for FreeBSD (attached here)
3. Fixing up header files for zipsync (attached here)
4. Modifying the CMakeLists.txt to use FreeBSD system libraries instead of the pre-built libraries for Linux (working, but still cleaning this up before upstreaming the changes)

I hope the portability changes will be accepted upstream. I'll also look to see which CMakeLists.txt changes would be appropriate to upstream, but it is not too much of a problem for me to maintain it locally for FreeBSD.

Once most of the changes have landed, I also plan to create a package ("port") with a thedarkmod and tdm_installer binary.

Regards,

Kevin

patch-FreeBSD2.diff patch-zipsync.diff

[thebigh]

Hi! Welcome to the community, and good work on getting TDM to run.

Have you had any luck with DarkRadiant as well?

[freyk]

Thanks for the port!

Is it possible that you publish (and maintain) these packages in popular bsd app repo's? (Like people did/tried with in some other package Operating-system app repo's)

For more info see TDM-wiki article:

https://wiki.thedarkmod.com/index.php?title=Installer_and_Manual_Installation

 

Edited April 10, 2022 by freyk

[stgatilov]

I applied the first patch with minor changes in svn rev 9890.

Regarding the second one:

Why did you add cast to (const uint8_t *) in Hasher::Update?
It seems that blake2s_update accepts const void * regardless of platform...

UPDATE: It turns out that older versions of BLAKE2 received const uint8_t * (commit).

[stgatilov]

Applied the second commit as svn rev 9892.

Also, I accidentally noticed that tdm_installer build is now broken because I changed the name of curl cmake package, so I changed that. We take libs from conan, so we have to use its convention: now we do "find_package(CURL)" everywhere.

[stgatilov]

Speaking of CMakeLists changes, you can minimize the changes on your side, then share the patch.

I'll see what you had problems with. Maybe I'll find a better way to fix something, maybe I'll apply it "as is" under appropriate option.

[Partmedia]

thebigh, I haven't looked at porting DarkRadiant yet. I'll likely take a look soon.

freyk, yes, I intended to package and maintain The Dark Mod (and tdm_installer) for FreeBSD. Is there a preferred package name for packages (tdm, thedarkmod, darkmod, or something else)?

Thanks, stgatilov, for reviewing and committing the patches. I'm working on reducing the CMakeLists diff and will provide an update once things are looking good.

[Partmedia]

A few notes and questions about the CMakeLists.txt:

• set(CMAKE_CXX_STANDARD 14) is already specified, so is "-std=c++14" required in add_compile_options?
• There are a few CMake modules for which system files are available under different names than those used by The Dark Mod. If we use the same names as CMake, it would make finding system versions of the libraries easier simply by removing the ThirdParty CMAKE_MODULE_PATH. The .cmake files in ThirdParty/cmake_find_package would need to be renamed, and the flags they set would also need to be adjusted to match those in the system CMake modules. I can provide a patch if this would be acceptable.
  • openal -> OpenAL
  • libjpeg -> JPEG
  • libpng -> PNG
  • zlib -> ZLIB
• There are other modules for which CMake does not provide system modules, for which I will grab in a OS-dependent way for my port (PkgConfig for me). Would there be interest in doing something like:

  if (USE_SYSTEM_LIBRARIES)
      include("${CMAKE_SOURCE_DIR}/ThirdParty/UsePkgConfig.cmake") # PkgConfig stuff in this file
  else()
  	# find third-party package modules
      list(PREPEND CMAKE_MODULE_PATH "${CMAKE_SOURCE_DIR}/ThirdParty/cmake_find_package")
  endif()

  Otherwise I'd just maintain this for the FreeBSD port.

Edited April 10, 2022 by Partmedia
Add missing ZLIB

[stgatilov]

Oh...

I thought pkgconfig is something installed as system package, which knows how to find many libraries by their standard name. If you have to write this UsePkgConfig.cmake yourself, and enumerate all libs there, then there is absolutely no point in using it: why not do the same with cmake?

[Partmedia]

9 minutes ago, stgatilov said:

Oh...

I thought pkgconfig is something installed as system package, which knows how to find many libraries by their standard name. If you have to write this UsePkgConfig.cmake yourself, and enumerate all libs there, then there is absolutely no point in using it: why not do the same with cmake?

pkgconfig is exactly that. My UsePkgConfig.cmake would probably only contain:

find_package(PkgConfig REQUIRED)
pkg_check_modules(minizip REQUIRED minizip)
pkg_check_modules(ogg REQUIRED ogg)
pkg_check_modules(vorbis REQUIRED vorbis vorbisfile)
pkg_check_modules(ffmpeg REQUIRED libavcodec libavdevice libavfilter libavformat libswresample libavutil libpostproc libswscale)
pkg_check_modules(pugixml REQUIRED pugixml)
pkg_check_modules(glfw REQUIRED glfw3)
pkg_check_modules(devil REQUIRED IL)

Which could easily just be included in the main CMakeLists.txt.

[stgatilov]

11 minutes ago, Partmedia said:

pkgconfig is exactly that. My UsePkgConfig.cmake would probably only contain:

Is it true that pkg-config provides more libraries or does so more accurately than cmake's built-in find_package? (at least on Unix-like systems)

[Partmedia]

8 minutes ago, stgatilov said:

Is it true that pkg-config provides more libraries or does so more accurately than cmake's built-in find_package? (at least on Unix-like systems)

I can't speak for all Unix-like systems, but in my experience packaging software on FreeBSD, pkg-config provides more libraries than CMake's built-in modules.

Either the library author or the packager will write a .pc file containing the flags necessary to compile and link to the library. When that library is installed, the .pc file is copied to a well-known directory (/usr/local/libdata/pkgconfig on FreeBSD). I haven't yet come across .pc file that had any incorrect system flags or broken linker flags.

Contrast this with CMake's system modules, which are bundled with CMake releases. Library detection for which system modules are not available fall to those who are shipping CMakeLists.

The disadvantage of pkg-config is that it will usually only look for .pc files installed in system-wide locations, which means that it will be harder to find libraries in user-installed locations. CMake modules are supposed to have a flag that teach CMake where to try to find the module, which should let it detect user-installed libraries. In practice, support for this varies on the quality of the CMake module.

[Partmedia]

So to conclude, if there is interest in providing a USE_SYSTEM_LIBRARIES flag, it could be implemented as:

if (USE_SYSTEM_LIBRARIES) # and on a platform on which pkgconfig is supported
  # do everything using pkgconfig
  find_package(PkgConfig REQUIRED)
  pkg_check_modules(...)
else()
  # do what we currently do
  list(PREPEND CMAKE_MODULE_PATH "${CMAKE_SOURCE_DIR}/ThirdParty/cmake_find_package")
  find_package(...)
endif()

[stgatilov]

3 hours ago, Partmedia said:

So to conclude, if there is interest in providing a USE_SYSTEM_LIBRARIES flag, it could be implemented as:

if (USE_SYSTEM_LIBRARIES) # and on a platform on which pkgconfig is supported
  # do everything using pkgconfig
  find_package(PkgConfig REQUIRED)
  pkg_check_modules(...)
else()
  # do what we currently do
  list(PREPEND CMAKE_MODULE_PATH "${CMAKE_SOURCE_DIR}/ThirdParty/cmake_find_package")
  find_package(...)
endif()

I assume pkg-config defines the same variables like openal_INCLUDE_DIRS and openal_LIBRARIES ?

Since some packages have different names, the pkg-config path needs to do set a few cmake variables too, so that the rest of the CMake file could use conan names.

And the last question is: whether it should be put straight into CMakeLists.txt, or it should be put into ThirdParty/cmake_find_package/FindXXX.cmake. The second option is probably better, but means a bit more bolierplate (basically the same "if" and the same "include" in every "find" file).

[Partmedia]

18 hours ago, stgatilov said:

I assume pkg-config defines the same variables like openal_INCLUDE_DIRS and openal_LIBRARIES ?

Since some packages have different names, the pkg-config path needs to do set a few cmake variables too, so that the rest of the CMake file could use conan names.

The variable names can differ from the package names, for example:

pkg_check_modules(FOO REQUIRED minizip)

Puts the minizip flags into FOO_INCLUDE_DIRS and FOO_LIBRARIES. Even if they are different, some manual variable hacking in CMakeLists would be possible and probably not too bad.

18 hours ago, stgatilov said:

And the last question is: whether it should be put straight into CMakeLists.txt, or it should be put into ThirdParty/cmake_find_package/FindXXX.cmake. The second option is probably better, but means a bit more bolierplate (basically the same "if" and the same "include" in every "find" file).

I would err on the side of making the changes as unintrusive to the existing code as possible. I'm not familiar with how TDM handles dependencies in Conan, or how it finds dependencies on other platforms (Win32, OS X), so I would defer to others.

[stgatilov]

Can you at least provide the pkg-config code that you use?
Preferably for all third-party libs used, except for probably header-only ones.

[freyk]

And about the package name, opensuse package creators uses "thedarkmod".

And to avoid several user problems, please set the folder permissions so all users can write to that folder. (Because people here dont want TDM to be system-wide)
(and dont want to follow fileplacement standards, that are dear to your Freebsd users)

 

Edited April 14, 2022 by freyk

[daren sain]

That's great to hear! You have done a fantastic job porting The Dark Mod to FreeBSD. Your hard work on fixing the build with Clang, modifying header files and CMakeLists.txt, and creating a package will greatly benefit the FreeBSD community. Keep up the good work and best of luck in getting your changes accepted upstream.

[Partmedia]

Hi folks, I'm finally coming back to this after some time

I've found that the latest SVN revision (10623 at the time of writing) is broken on FreeBSD 13.2/clang 14.0.5 again, but am preparing patches to fix them. Some of these issues include:

• Printing strings in `game/LodComponent.cpp` missing `.c_str()`
• INFINITY is a system-wide macro in <cmath> that conflicts with idMath::INFINITY
  • Is renaming it to ID_INFINITY the right move here?
• ALIGN is a system-wide macro that conflicts with ALIGN in `idlib/sys/sys_defines.h`
  • I plan to rename the macro to something like ID_ALIGN to avoid the conflict
• The `register` keyword is not allowed in C++17

While I prepare patches for these issues, what's the best way of providing patches upstream? As a patch upload here, or something else?

I've also noticed that there is now a Git mirror (thank you), which makes it easier for me to prepare local commits and separate different patches from one another for purposes of upstreaming.

[Daft Mugi]

5 minutes ago, Partmedia said:

While I prepare patches for these issues, what's the best way of providing patches upstream? As a patch upload here, or something else?

I personally use GitHub gists to share patches for review. That way I can update them more easily.

I format the patch so that it is SVN compatible. I use git locally, so I run the command:

git diff --no-prefix > r0000-my-patch-name.diff

"r0000" prefix is the SVN revision it is based on. For example, https://gist.github.com/daftmugi/41d0324107e8734f364bb3e50ff00794

But perhaps @stgatilov has another suggestion?