Thelvyn's Thread

[thelvyn]

If we are adding all this other code (As we are) and we need to implement things beyond what the sdk offers to us (which we do) why not consider boost for things like this ?

 

Instead of having to implement it ourselves on multiple OS we could use boost for it and forget about whether this will work on linux or mac down the road. We will already know that it will work without changing anything down the road.

[sparhawk]

Maybe you could start by explaining what boost is supposed to be. Obviously we want to keep OS specific stuff to a minimum. Intrudicing a library that handles such stuff sounds good as long as it really works good on all platforms. Otherwise we have the same problem as before only with additional overhead.

[Crispy]

It's a collection of C++ libraries for doing just about every common task you could imagine (and some less common ones), in the spirit of the STL but with lots more optional modules.

 

http://boost.org/

 

It might make sense to use it. Worth investigating at least.

[OrbWeaver]

Just to weigh in here, Boost absolutely rocks. I brought it into DarkRadiant a while ago and it provides loads of useful features like smart pointers, regular expressions, improved string processing, more advanced containers, generic function objects, type-safe generic variables and loads more which I haven't even looked at yet.

[thelvyn]

It was started by members of the library working group(C++ Standards committee) but thousands of programmers world wide work on it. Getting a library into boost is hard. Their review process is brutal

I am subscribed to the mailing list, well the dev one anyway. There is also a user mailing list I believe.

 

Check out http://www.boost.org for more information.

 

I also subscribe to the wxwidgets dev mailing list which is even more active then boost-dev is.

[sparhawk]

I usually use wxWidgets for all my projects and it is really great to use, even though it could use more docs. Never heard about boost though. Sounds interesting. Ever since I started with Java I was impressed with how many objects there are and they all work well together, which is sadly lacking in C++ despite it's OO claim. Boost sounds quite like this.

 

Does boost and wxWidgets go together? Would this even make sense? Seems boost offers quite a lot while wx has some other stuff that I haven't seen in boost (only took a acursory glance right now) like databases for instance.

[OrbWeaver]

Does boost and wxWidgets go together? Would this even make sense? Seems boost offers quite a lot while wx has some other stuff that I haven't seen in boost (only took a acursory glance right now) like databases for instance.

 

Theoretically you should be able to use as many such libraries as you wish, providing they don't do something stupid like overload operator new(), or you do something stupid like try to use two different shared pointer classes to manage one object.

[thelvyn]

Boost has much better documentation, easier to use and less bugs IMHO after having used both of them.

I tend to favor boost over wxwidgets classes whenever possible as a result.

Many of boost classes do not require you to link to their libraries either. Like their smartptr classes are nice.

 

I would like to use those already but have not done so as anyone who does not have boost on their system would not be able to compile it.

 

If you decide it is permissible to use it we will need to setup a "Installing and Configuring BOOST" thread probably as it is not very intuitive. They have their own build system.

 

A little work for great benefits down the road though seems like a fair trade to me

[OrbWeaver]

If you decide it is permissible to use it we will need to setup a "Installing and Configuring BOOST" thread probably as it is not very intuitive. They have their own build system.

 

What I did for DarkRadiant was to import the Boost headers and those compiled libraries that are needed (only 2 so far, everything else is header-only) into the source tree, so individual developers wouldn't have to obtain and build Boost separately. This is a little redundant for the Linux users (me), but it means the tree can just be checked out and compiled without having to find extra libraries on Windows.

[sparhawk]

CVS should always compile out of the box, with no extra stuff to setup. So we have to include all neccessary files that re required for programmers to compile.

[thelvyn]

Does anyone know of a reason for the dll to fail to load ? I do not think its anything I did but I could be wrong

[sparhawk]

Most likely it is that **** assert. Use the logging (without ifdef DEBUG) and remove the asserts. asserts only work in debug builds and if users have problems, they can not even tell us what's wrong. That's why we have the logging mechanism, because it can be turned on on the client site as well.

[thelvyn]

Actually no thats not it. I forgot about devil.dll I just now successfully ran the mod.

JIT pops up on an assert instead of just failing to load so I would have known that.

[sparhawk]

Still, remove the assert and replace it with an error class log entry.

 

LT_SYSTEM, LC_ERROR.

[thelvyn]

I have to ask. Should I remove the other 1010 assert statements littered throughout the code also

 

I use assert on a must have item while debugging. If it works its always gonna work or I would implement it differently.

 

I need some way of having the debugger kick in there if it happens how do you suggest I do this if I do not use assert ?

[sparhawk]

Yeah. You can leave in the assert, but still accompany it with an error log where appropriate.

[thelvyn]

Yep yep will do. Logging is definitely in. Have to add something to flush it when there is a problem though.

 

Right now if I use exit or abort (Instead of assert I am trying ) the log file is not being saved.

 

I was thinking of adding a function to the CGlobal class that will flush and close the log file then call abort.

The file* is public and accessible but better to keep it in the class I think.

 

As far as aborting goes I do not think it is advisable to try and raise a dialog or message box from there but we should look into doing something so we can explain if we have to abort or something WHY it is crashing so people aren't left wondering. Doom3 is very resistant to giving up full screen mode to allow anything else to show is why I believe a regular dialog or message box would be inadvisable.

I will give it a try and see in the meantime but I am not hopeful.

Naturally this would only be in a case where we cannot continue or degrade gracefully.

 

Input ?

[sparhawk]

IMO logging is always flushed. At least it should be. This should be pretty simple, as the outbuffer just have to be set to NULL for the FILE.

 

As for explaining the crash. Question is what the user really can do with it. In most cases he wont have a clue and if the problem persists, he should switch on the logfile, so that he can look at whats causing it, or send it to us for analyzing it. That's what the log is for.

[thelvyn]

If you have been reading the other thread where I am active(Keyboard handle != mouse) you probably know that I implemented a api hook.

 

It seems we cannot use it after all since what I wanted it for is apparently not hookable at all.

 

So is there anything we would like more access to possibly ?

Usually this would be windows api calls and the like.

 

You get them to call your code first, then if you like you modify the arguments, call the original and optionally change the return code or data if pertinent as well.

 

If not I will remove the code completely.

[sparhawk]

I don't think we need that, if it doesn't work for that particular issue. We shouldn't use this to extensivly anyway, as it would just add more portabillity issues. I would be interested to see how you did the hooking though. I once did this with the help of assembly and injecting code with the VirtualWrite/Read functions. Woudl be interesting to see how you did it.

 

We could use that method to force hook on that Id function, but I don't really want to use such a method, as it can also creates a lot of problems. It needs a dissasembler to work properly, which I also wrote for that purpose.

[thelvyn]

It is pretty ugly right now, the code that is. I will clean it up and email it to you if you like but it is a bit complicated.

[sparhawk]

No prob.

[thelvyn]

I am not using this but this should be working injector code.

const char* sharedDll = "sharedmodule.dll";
const int sd_len = strlen( sharedDll );

int DoHook(int pid, bool UnHook, HMODULE hFreeModule)
{

HANDLE hProcess = ::OpenProcess(PROCESS_ALL_ACCESS, // Specifies all possible access flags
								FALSE, 
								pid);
if (hProcess == NULL) 
	return 0;

//	char szLibFile[MAX_PATH];

//	int cch = 1 + strlen(szLibFile);

PSTR pszLibFileRemote = (PSTR)::VirtualAllocEx(hProcess, NULL, sd_len, MEM_COMMIT, PAGE_READWRITE);

if (pszLibFileRemote == NULL) 
{
	printf("pszLibFileRemote was NULL");
	return 0;
}

if (!::WriteProcessMemory(hProcess, (PVOID)pszLibFileRemote, (PVOID)sharedDll, sd_len, NULL))
{
	printf("\nWriteProcessMemory Failed");
	return 0;
}

PTHREAD_START_ROUTINE pfnThreadRtn = NULL;

if(UnHook)
	pfnThreadRtn = (PTHREAD_START_ROUTINE)::GetProcAddress(::GetModuleHandle("Kernel32"), "FreeLibrary");
else
	pfnThreadRtn = (PTHREAD_START_ROUTINE)::GetProcAddress(::GetModuleHandle("Kernel32"), "LoadLibraryA");

if (pfnThreadRtn == NULL) 
{
	printf("\nGetProcAddress Failed");
	return 0;
}

HANDLE hThread;

if(UnHook)
	hThread = ::CreateRemoteThread(hProcess, NULL, 0, pfnThreadRtn, (HMODULE)hFreeModule, 0, NULL);
else
	hThread = ::CreateRemoteThread(hProcess, NULL, 0, pfnThreadRtn, (PVOID)pszLibFileRemote, 0, NULL);

if (hThread == NULL) 
{
	printf("\nCreateRemoteThread Failed");
	return 0;
}

::WaitForSingleObject(hThread, INFINITE);

if (pszLibFileRemote != NULL) 
  ::VirtualFreeEx(hProcess, (PVOID)pszLibFileRemote, 0, MEM_RELEASE);

if (hThread  != NULL) 
	::CloseHandle(hThread);

if (hProcess != NULL) 
	::CloseHandle(hProcess);

return 1;
}

// We will require this function to get a module handle of our
// original module

HMODULE EnumModules(int pid, char szLibFile[MAX_PATH])
{
HMODULE hMods[1024];
DWORD cbNeeded;

unsigned int i;

HANDLE hProcess = ::OpenProcess(PROCESS_ALL_ACCESS, // Specifies all possible access flags
								FALSE, 
								pid);

if (hProcess == NULL) 
	return 0;

HMODULE m_hModPSAPI = ::LoadLibraryA("PSAPI.DLL");

typedef BOOL (WINAPI * PFNENUMPROCESSMODULES)
(
	HANDLE hProcess,
	HMODULE *lphModule,
	DWORD cb,
	LPDWORD lpcbNeeded
);

typedef DWORD (WINAPI * PFNGETMODULEFILENAMEEXA)
(
	HANDLE hProcess,
	HMODULE hModule,
	LPSTR lpFilename,
	DWORD nSize
);

PFNENUMPROCESSMODULES   m_pfnEnumProcessModules;
PFNGETMODULEFILENAMEEXA m_pfnGetModuleFileNameExA;

m_pfnEnumProcessModules = (PFNENUMPROCESSMODULES)::GetProcAddress(m_hModPSAPI, "EnumProcessModules");

m_pfnGetModuleFileNameExA = (PFNGETMODULEFILENAMEEXA)::GetProcAddress(m_hModPSAPI, "GetModuleFileNameExA");

if( m_pfnEnumProcessModules(hProcess, hMods, sizeof(hMods), &cbNeeded))
{
	for ( i = 0; i < (cbNeeded / sizeof(HMODULE)); i++ )
	{
		char szModName[MAX_PATH];

		// Get the full path to the module's file.

		if ( m_pfnGetModuleFileNameExA( hProcess, hMods[i], szModName, sizeof(szModName)))
		{
			// Print the module name and handle value.

			printf("\t%s (0x%08X)\n", szModName, hMods[i] );

			if(strcmp(szModName, szLibFile) == 0)
			{
				::FreeLibrary(m_hModPSAPI);

				return hMods[i];
			}
		}
	}
}

if (hProcess != NULL) 
::CloseHandle(hProcess);

return 0;
}

 

Its not beautified and using some globals at the moment but gives you the idea anyway.

Its a WIP although it was working last time I tried it but I have made some changes and have not tested it

lately.

 

I havent used this since I switched from visual studio 6 enterprise edition. I went directly to 2005 maybe a year ago. So it should still work.

[thelvyn]

Ok here is what I came up with for hooking getmessage from our DLL.

It works, I was trying to make a class to make it simpler but for some reason it seemed to be losing its calling convention using a typedef with a template. Well it just wasn't working and I figured it wasn't worth investing any more time in it since we aren't even going to be using it now.

 

#include "../idlib/precompiled.h"
#pragma hdrstop

static bool init_version = FileVersionList("$Source: /cvsroot/darkmod_src/DarkMod/apihook.cpp,v $  $Revision: 1.0 $   $Date: 2006/12/29 07:45:07 $", init_version );

#include "DarkModGlobals.h"
#include <imagehlp.h>
#include <Tlhelp32.h>
#include <stdio.h>
#include "msi_exception.hpp"

/****************************************
****************************************
*
*******************************

	This module must be in the same address space as the intended target in order to work. There
are several methods you can use to do this but that is beyond the scope of what I am doing here.

	As the darkmod dll is already in the address space of doom3 this makes it much simpler to implement
as we do not need to worry about injecting a dll.

****************************************
****************************************
*
*******************************/

// Declare the signature for our replacement functions here


typedef BOOL WINAPI GetMessage_t( LPMSG lpMsg, HWND hWnd, UINT wMsgFilterMin, UINT wMsgFilterMax );
BOOL WINAPI AH_GetMessageA( LPMSG lpMsg, HWND hWnd, UINT wMsgFilterMin, UINT wMsgFilterMax );
BOOL WINAPI AH_GetMessageW( LPMSG lpMsg, HWND hWnd, UINT wMsgFilterMin, UINT wMsgFilterMax );


PROC EnumAndSetHooks( LPSTR BaseLibraryName, LPSTR BaseFunctionName, PROC NewFunctionPointer, bool UnHook, PROC Custom );
void SetHook(HMODULE hModuleOfCaller, LPSTR LibraryName, PROC OldFunctionPointer, PROC NewFunctionPointer);

// to do: Move all this to a class instead
namespace { // only accessible from this file
HANDLE				g_CurrentHandle;
GetMessage_t * GetMessageA_Orig = NULL;
GetMessage_t * GetMessageW_Orig = NULL;
}

BOOL APIENTRY DllMain( HANDLE hModule, DWORD  ul_reason_for_call, LPVOID lpReserved )
{
switch (ul_reason_for_call)
{
	case DLL_PROCESS_ATTACH:
		// we dont need thread messages so disable them
		::DisableThreadLibraryCalls( (HMODULE)hModule );
		// set up structured exception handling
		CStructuredExceptionInit();
		// save module handle
		g_CurrentHandle = hModule;
		GetMessageA_Orig = (GetMessage_t*)EnumAndSetHooks( "user32.dll", "GetMessageA", (PROC) AH_GetMessageA, false, 0 );
		GetMessageW_Orig = (GetMessage_t*)EnumAndSetHooks( "user32.dll", "GetMessageW", (PROC) AH_GetMessageW, false, 0 );				
		break;
	case DLL_PROCESS_DETACH:
		// Unhook us
		EnumAndSetHooks("user32.dll", "GetMessageA", (PROC) GetMessageA_Orig, true, (PROC) AH_GetMessageA );
		EnumAndSetHooks("user32.dll", "GetMessageW", (PROC) GetMessageW_Orig, true, (PROC) AH_GetMessageW );
		break;
}
return TRUE;
}

BOOL WINAPI AH_GetMessageA( LPMSG lpMsg, HWND hWnd, UINT wMsgFilterMin, UINT wMsgFilterMax )
{
Debug( "AH_GetMessageA Called" );
BOOL rc = GetMessageA_Orig( lpMsg, hWnd, wMsgFilterMin, wMsgFilterMax );
return rc;
}

BOOL WINAPI AH_GetMessageW( LPMSG lpMsg, HWND hWnd, UINT wMsgFilterMin, UINT wMsgFilterMax )
{
Debug( "AH_GetMessageW Called" );
BOOL rc = GetMessageW_Orig( lpMsg, hWnd, wMsgFilterMin, wMsgFilterMax );
return rc;
}

void SetHook(HMODULE hModuleOfCaller, LPSTR LibraryName, PROC OldFunctionPointer, PROC NewFunctionPointer)
{
if( hModuleOfCaller == g_CurrentHandle || hModuleOfCaller == 0 )
{
	return;
}
ULONG ulSize;
// Get the address of the module’s import section
PIMAGE_IMPORT_DESCRIPTOR pImportDesc = (PIMAGE_IMPORT_DESCRIPTOR) ImageDirectoryEntryToData( hModuleOfCaller, TRUE, IMAGE_DIRECTORY_ENTRY_IMPORT, &ulSize );
if( pImportDesc == NULL ) // Does this module have an import section ?
{
	return;
}
// Loop through all descriptors and find the 
// import descriptor containing references to callee’s functions
while( pImportDesc->Name )
{
	PSTR pszModName = (PSTR)((PBYTE) hModuleOfCaller + pImportDesc->Name);

	if( stricmp( pszModName, LibraryName ) == 0 )
	{
		break; // Found
	}
	pImportDesc++;
} // while
if( pImportDesc->Name == 0 )
{
	return;
}
//Get caller’s IAT
PIMAGE_THUNK_DATA pThunk = (PIMAGE_THUNK_DATA)( (PBYTE) hModuleOfCaller + pImportDesc->FirstThunk );
PROC pfnCurrent = OldFunctionPointer;
// Replace current function address with new one
while( pThunk->u1.Function )
{
	// Get the address of the function address
	PROC* ppfn = (PROC*) &pThunk->u1.Function;
	// Is this the function we’re looking for?
	BOOL bFound = (*ppfn == pfnCurrent);
	if (bFound)
	{
		MEMORY_BASIC_INFORMATION mbi;

		::VirtualQuery(ppfn, &mbi, sizeof(MEMORY_BASIC_INFORMATION));

		// In order to provide writable access to this part of the 
		// memory we need to change the memory protection

		if (FALSE == ::VirtualProtect(mbi.BaseAddress,mbi.RegionSize,PAGE_
READWRITE,&mbi.Protect))
		{
			return;
		}

		*ppfn = *NewFunctionPointer;

		BOOL bResult = TRUE;

		// Restore the protection back
		DWORD dwOldProtect;

		::VirtualProtect(mbi.BaseAddress,mbi.RegionSize,mbi.P
rotect,&dwOldProtect);

		break;
	} // if

	pThunk++;

} // while
}

PROC EnumAndSetHooks( LPSTR BaseLibraryName, LPSTR BaseFunctionName, PROC NewFunctionPointer, bool UnHook, PROC Custom )
{
HMODULE hMods[1024];
DWORD cbNeeded;
unsigned int i;
typedef BOOL (WINAPI * PFNENUMPROCESSMODULES) (	HANDLE hProcess, HMODULE *lphModule, DWORD cb, LPDWORD lpcbNeeded );
HMODULE hBaseLib = LoadLibrary(BaseLibraryName);
PROC hBaseProc;
if( UnHook )
{
	hBaseProc = (PROC) Custom;
}
else
{
	hBaseProc = GetProcAddress(hBaseLib, BaseFunctionName);
}
PFNENUMPROCESSMODULES m_pfnEnumProcessModules;
HMODULE m_hModPSAPI = ::LoadLibraryA( "PSAPI.DLL" );
m_pfnEnumProcessModules = (PFNENUMPROCESSMODULES)::GetProcAddress(m_hModPSAPI, "EnumProcessModules");
HANDLE hProcess = ::GetCurrentProcess();
if( m_pfnEnumProcessModules(hProcess, hMods, sizeof(hMods), &cbNeeded))
{
	for ( i = 0; i < (cbNeeded / sizeof(HMODULE)); i++ )
	{
		SetHook(hMods[i], BaseLibraryName, hBaseProc, NewFunctionPointer); 
	}
}
return hBaseProc;
}

 

Well there you have it. Works nicely

Whatever you do make sure you get the calling convention correct on whatever function you are hooking or you will crash, maybe not the first time or even the second but you will crash.

[sparhawk]

Ah, thanks! That was the function I was also using. These Virtual*. I thought meaybe there is another way that I didn't knew of. What I find interesting though is that you are just overwriting the pointer of the function. Is this always a pointer TO the function? Because if it is the adress of the function itself, and you overwrite it like that, then this would cause a crash.

 

What I did was, to find the address of the function. Then dissassembled the first N bytes (how much I needed to replace it). Then I replaced the original function with a jmp instruction to my own, and copied the original instructions to some other place. Then I put another jump instruciton behind it, point back directly after my inserted jmp. The advantage is that this will always work, even for functions that are not accessed via a jumptable, but of course it requires a dissassembler to work, because you need to know how many bytes you must copy.