Hacking Doom3 closed source

[stgatilov]

From time to time most of coders have to dig in the Doom3 closed source. I hope it won't be necessary from the next year, but it is still viable today. I feel that most of TDM coders already know asm well, so they can use some disassembler/debugger to look through Doom3.exe. What tools/ways do you use to look Doom 3 executable?

 

I use ollydbg2 for it. It is user-level debugger with good analyzing capabilities and convenient UI. It is capable of showing debug symbols, so I can see TDM and GL functions clearly. Also it has a lot of helpful searching routines like "search for all referenced strings" of "find references to". Of course it allows to set breakpoints on code/memory (including conditional ones), moreover it allows code patching with assembler at runtime.

 

I've seen IDA before. IDA is very good analyzing disassembler, but it is not designed for debugging. And analyzing the code statically is much less powerful that debugging it.

 

 

 

 

 

[Aprilsister]

That's cool. Are you using the alpha 2.01 or sticking with 2.0?

Edited January 30, 2011 by Aprilsister

[stgatilov]

That's cool. Are you using the alpha 2.01 or sticking with 2.0?

Of course I use 2.01 alpha 1 (the latest one I think). Pure 2.0 version cannot load debug symbols, so I'd downloaded 2.01 as soon as I saw it.

There is no reason to stick with older version of software like olly: It is free, it is developed by one person, it is provided "as is" and can be buggy anyway. Sticking with old version won't help you

 

 

[Aprilsister]

Sounds good enough. Been messing with the 2.01 anyway while waiting for your answer, seen nothing disaterous... but haven't exactly pushed any boundaries (to say the least).

 

Very nice default layout and general organization of the thing. Thanks for the recommendation, though I realize you were actually asking for recommendations.

[stgatilov]

Thanks for the recommendation, though I realize you were actually asking for recommendations.

Not really I was asked how I looked Doom3 closed code. So I decided to start a thread about it. To be honest, I doubt that anyone can offer a better tool that olly, but there is some chance... Also I was interested in what tools are popular. IDA is mentioned quite often, maybe some hardcore debuggers prefer SoftICE? Also there is windbg which was a prototype for olly AFAIU...

[Tels]

hardcore debuggers prefer SoftICE?

 

Er, I am trying as hard as possible to forget that thing with soft in the name...

 

What *I* *could* do is using a hardware debugger (basically a re-wried CPU), but A: I don't think anyone else has got one, anyway, they are expensive, and B: work-equipment using for a personal project is a no-no anyway and C: I am not sure if would really give much more insight.

 

I haven't even tried using gnu debugger under linux, has someone experiences with it?

[Aprilsister]

work-equipment using for a personal project is a no-no

 

Never fear. Come The Revolution, we will seize the means of production!

 

For truly productive means!

[Serpentine]

I mostly just use IDA, since most of the stuff I do works on analysis rather than fixing That said I've never really tried debugging with it, and while I think the python interface and such could make that... 'interesting' it does seem a bit clunky. AMD's PerfStudio works quite well for looking at the OpenGL side of things, nvidia's tools kept crashing and messing with drivers and gDEBugger wouldnt attach at all.

 

That said the mac binaries provide a load of info that's stripped from the windows binaries, which also helps with generating detailed graphs which are quite readable and psuedo code generation which doesn't make the eyes want to pop (hey, it's fun :V)

[stgatilov]

That said the mac binaries provide a load of info that's stripped from the windows binaries, which also helps with generating detailed graphs which are quite readable and psuedo code generation which doesn't make the eyes want to pop (hey, it's fun :V)

What sort of info is it? The code just looks well or it contains names (like symbols or other text hints)?

[stgatilov]

AMD's PerfStudio works quite well for looking at the OpenGL side of things, nvidia's tools kept crashing and messing with drivers and gDEBugger wouldnt attach at all.

 

Emm... Could you write what sort of info the GL tools can provide? For example what can you learn from running PerfStudio on Doom3+TDM ?

[Serpentine]

Emm... Could you write what sort of info the GL tools can provide? For example what can you learn from running PerfStudio on Doom3+TDM ?

 

I'm not too sure what you'd be interested in, but it gives a fairly good breakdown of frames, offering a good bit of interactive profiling as well as facilities for browsing through the process of frame generation. It's mildly interesting, I thinkits mostly of importance if you're looking at performance or debugging visual things. I havent looked at the shader debugging at all, as its something I'm not very good with I found it handy for checking textures in memory to better understand the resource management and for figuring out if different material shaders were slower/faster when updating water and such. It's not super complex and it's free...

http://developer.amd.com/gpu/PerfStudio/Pages/default.aspx

[stgatilov]

That said the mac binaries provide a load of info that's stripped from the windows binaries, which also helps with generating detailed graphs which are quite readable and psuedo code generation which doesn't make the eyes want to pop (hey, it's fun :V)

Could you share the way you watch the call graph? I've generated the call graph, but its size is so huge that any graph visualizer is slow as hell and completely useless. It must be simple to draw f.i. all vertices within distance 2/3 of single vertex, but I cannot find that simple functionality...