Multiplying Md5anims

[ascottk]

I think something needs to be done (it's partially my fault) about the md5anims. There are 46 md5anim files in the citywatch folder weighing in at 2.86 megs. Now multiply that number by the AIs who "use" those animations. Let's see:

1 citywatch

1 builder priest

1 nobleman

1 houseguard

1 merc proguard

 

That's nearly 18 megs of data for the same animations

 

My idea is (this is going to take a lot of work, but not as bad as separating meshes, detaching heads, rerigging, etc.) to have two different sizes, then point one size to one md5anim directory then the other to another directory. This means we'll have less variances in sizes & we'll have to be extra careful with rigging. Instead of 18 megs we can bump it sown to nearly 6.

 

Another thought, I'm not sure how well these files compress for distribution (just checked, the citywatch goes down to 1,097 kb). The only problem with the disk space is uncompressed. Is it worth using the same md5anims?

____________

 

You might ask why we weren't using the same md5anims in the first place. As I explained in other threads, rigging is tricky & often times Doom 3 imports the md5anims with joints I did not want. Thus one AI's md5anim may not be compatible with another's md5mesh because the joint numbers or names won't match. There's also the issues with the size of the AI. With our current setup our AI vary in size which is a good thing. Another issue is, if an md5anim is shared, would the AI react differently or the same? If one scratches his nose will another do it at the same time?

[New Horizon]

The shared directory is a great idea, and one I highly endorse if it can be done without any major drawbacks. The majority of T3 animations were shared this way I think. I find doom 3 to be inefficient in this regard and I think we could definitely make some improvements.

 

Hopefully this will be possible. I don't think AI sharing these animations would play the anims at the same time, otherwise two city watch elite guards would be doing the same already.

[Springheel]

My idea is (this is going to take a lot of work, but not as bad as separating meshes, detaching heads, rerigging, etc.) to have two different sizes

 

Hmmm. This would mean resizing and rerigging all the AI again? And then redoing all the entityDef offsets, and heads? Eh. Not my favourite option.

 

Is the size really that big a deal? They seem to compress well, so downloading won't be huge issue.

 

If there are characters that can already share the same files because they have the same size/joints, then by all means have them share, but I'm very hesitant to support such a massive reorganization.

[ascottk]

Hmmm. This would mean resizing and rerigging all the AI again? And then redoing all the entityDef offsets, and heads? Eh. Not my favourite option.

Resizing is a no brainer any more (just add -scale x.x to the def file when exporting) so that's not a problem. Rerigging the AI are mainly touch ups so rerigging is a misnomer. The offsets? I don't think anyone spent time with D3 head offsets on the newer AI.

 

Is the size really that big a deal? They seem to compress well, so downloading won't be huge issue.

That's why I was wondering if all this would be worth it.

 

If there are characters that can already share the same files because they have the same size/joints, then by all means have them share, but I'm very hesitant to support such a massive reorganization.

That seems to be the case in this mod reorganization . . .

[OrbWeaver]

It does seem a lot of work, when you consider that the mod will be several gigs in size due to the textures and sounds, saving 12 megabytes would be pretty much irrelevant (and that is ignoring the fact that those 12 megabytes will be something a lot smaller when the text-based MD5 files are compressed).

[New Horizon]

I think it would also save some extra work for us with future animations. It also makes sense to have one set of animation files for all similar sized AI. Easier for FM authors to setup their own custom AI in the future too.

[Crispy]

From the space-saving point of view I agree it doesn't make much sense.

 

From the organisational point of view, that's really up to you to weigh up. Will it actually make things easier for you to maintain? If so, is that saving worth the up-front cost of re-organising everything?

[ascottk]

Back to this because the animations are piling up & it's getting difficult to update all of the AI. If a few animations are committed then we have to add those animations to all of the AI that share the citywatch animations. Quite a few are sharing those animations (houseguard, nobleman, engineer, citywatch, elite proguard, builder priest, and others I probably forgot) so it's getting messy. This means I have to point the def file to the new maya source files & add the animations to the model decls.

 

This is something I'll mull over for the next few weeks. If we could use one def files for exporting it'll be a lot easier. Maybe we could inherit some default settings & apply scaling & other custom stuff AI will need. I'm not sure it's possible but it's something to try.

[Domarius]

Well, whatever you work out, I'll do it. Anything that makes a process easier, I'm eager to do.

[ascottk]

Well, whatever you work out, I'll do it. Anything that makes a process easier, I'm eager to do.

Yeah, if we could use one def file for the animations then that'd be a helluva lot easier. Editing 10 different ones, even for 1 new animation, is a major PITA.

[Domarius]

You're talking about def file inheritance? That's doable. Def file inheritance is basically just a virtual copy and paste at the "inherit" line .

 

What kinds of scaling/rotating differences have the different models had for each anim?

 

If there are some, then we need to sort this out so that we CAN use the one def file with no custom scale/rotation tweaks for each model...

[ascottk]

You're talking about def file inheritance? That's doable. Def file inheritance is basically just a virtual copy and paste at the "inherit" line .

I haven't seen anything in the export section in the def file for inheritance & the maya->doom 3 plugin may not play by d3's rules of inheritance.

 

What kinds of scaling/rotating differences have the different models had for each anim?

The rotation thing is easy to do in maya but the scaling thing never quite worked for me. I haven't used any rotation options in a def file (except the thief, but he's using a totally different set of animations) so rotation is not an issue.

 

But the scaling may be the tricky part. I do like having different sizes per AI.

 

If there are some, then we need to sort this out so that we CAN use the one def file with no custom scale/rotation tweaks for each model...

I was thinking along the lines of this (inheriting citywatch):

export inventor_engineer {
inherit citywatch
options -sourcedir models/ask/inventor_engineer -destdir models/md5/chars/inventors/engineer -scale .92

Then the mesh & animations will be exported & tailored for the engineer. The options would include the joint specific data, unless I decide to match weighting for all of the AI which is extremely tricky.

 

Then in the modelDef:

model tdm_ai_inventor_engineer {
inherit tdm_ai_citywatch -dir models/md5/chars/inventors/engineer	
mesh					   models/md5/chars/inventors/engineer/tdm_ai_inventor_engineer.md5mesh
. . .

That way we only need to edit the citywatch's def file & it'll call the animations in the engineer's folder. I really doubt it'll work though.

[Crispy]

I don't see any way to do that except using preprocessor hacks, and I don't think .def files support preprocessor directives (#include, #define, and so on).

 

IIRC the parsing of .def files isn't in the SDK either. Scripts are though.

[Domarius]

Okay well... what if one of our awesome programmers can make some kinda little utility that automates the creation of multiple def files in this way?

 

Somehow it could take some template def files and propigate its contents into target def files, and the target ones keep their special differences.

 

So like in ascottk's example, the "inherit" line is replaced with the contents of that file it's trying to inherit, and a new file is created with the inherited stuff pasted in, and this is what we use to export. The original files with the inherit line are "working files" and the utility exports proper def files from them.

[Crispy]

Hmm. That's a bit hacky. I'd call that a last resort, but if there aren't any other ways then I guess we just have to do what we can.

 

@ascottk: If you can come up with a precise and detailed list of requirements (since I'm not sure exactly what is required - remember I'm a total n00b with model defs and the exporting process), I'll see about hacking together a Python script for you, along the lines that Domarius suggests.

[ascottk]

I don't think mass editing multiple def files is the way to go. The maya->doom 3 plugin source is in the sdk (we use maya 7 but the source for that version is nowhere to be found). I think the best way is to customize the plugin with extra options.

 

http://www.facinghell.com/?cid=files (maya 7 plugin)

http://www.modwiki.net/wiki/Export_%28decl%29 (modwiki)

http://www.iddevnet.com/doom3/modelexport.php

 

The main def file where other AI will inherit from:

export tdm_humanoid_animations {

options -sourcedir models/ask/citywatchMBs -destdir models/md5/chars/guards/citywatch -scale .94
mesh citywatchmesh.mb -dest tdm_ai_citywatch
addoptions -keep Hips LeftHips_Dummy LeftUpLeg LeftUpLegRoll LeftLeg LeftLegRoll LeftFoot LeftToeBase LeftToe_end joint3 joint4 RightHips_Dummy RightUpLeg RightUpLegRoll RightLeg RightUpLegRoll RightLegRoll RightFoot RightToeBase joint5 joint6 Spine_Dummy Spine Spine1 Spine2 LeftShoulder_Dummy LeftShoulder LeftArm_Dummy LeftArm LeftArmRoll LeftForeArm LeftHand LeftHandIndex1 LeftHandIndex2 LeftHandIndex3 LeftHandRing1 LeftHandRing2 LeftHandRing3 LeftHandRing4 LeftHandThumb1 LeftHandThumb2 LeftHandThumb3 Neck Head RightShoulder_Dummy RightShoulder RightArm_Dummy RightArm RightArmRoll RightForeArm RightForeArmRoll RightHand RightHandIndex1 RightHandIndex2 RightHandIndex3 RightHandMiddle1 RightHandMiddle2 RightHandMiddle3 RightHandThumb1 RightHandThumb2 RightHandThumb3 joint8 joint9 leftpad rightpad joint7 sword -scale .94
anim AnimRef.mb -dest af_pose.md5anim //-scale .94
//addoptions -scale 1

anim idle.mb //-scale .94
anim idle_torch.mb //-scale .94
anim idle_sword.mb //-scale .94

anim walk.mb
anim walk_torch.mb
anim run.mb
anim run_torch.mb

anim look_around.mb
anim drawsword.mb

anim attack.mb
anim attack2.mb
anim attack3.mb

anim search.mb
anim throw.mb

anim cower.mb
anim surprise_big.mb

anim painchest.mb
anim painhead.mb
anim painrightarm.mb
anim painleftarm.mb
anim death.mb

anim anim_SitIdle_00.mb -dest SitIdle.md5anim

/***Solis Anims***/
anim alertwalk.mb
anim blindwalk.mb
anim waistpain.mb
anim surprised.mb
anim failedKO.mb

/*************
* Bow animations
*************/

anim bow_fire.mb

/*************
* Animations for fun/jokes/whatever
*************/
anim dance1.mb
anim flip.mb
anim itch.mb
anim nod_head.mb
anim shake_head.mb

/***squill's anims***/
anim anim_LayDown_00.mb -dest LayDown.md5anim
anim anim_GetUp_00.mb -dest GetUp.md5anim
anim anim_SitDown_00.mb -dest SitDown.md5anim
anim anim_FailedKO_00.mb -dest failedKO2.md5anim

/***Doug's anims***/
anim armSniff1.mb -dest SniffArm1.md5anim
anim cough1.mb
}

 

Now we want to customize that for a certain AI:

export inventor_engineer {

options -sourcedir models/ask/inventor_engineer -destdir models/md5/chars/inventors/engineer -scale .92
mesh inventor_engineer.mb -dest tdm_ai_inventor_engineer
addoptions -keep Hips LeftHips_Dummy LeftUpLeg LeftLeg LeftFoot LeftToeBase joint3 joint4 RightHips_Dummy RightUpLeg RightLeg RightFoot RightToeBase joint5 joint6 Spine Spine1 Spine2 LeftShoulder_Dummy LeftShoulder LeftArm_Dummy LeftArm LeftArmRoll LeftForeArm LeftHand LeftHandIndex1 LeftHandIndex2 LeftHandIndex3 LeftHandRing1 LeftHandRing2 LeftHandRing3 LeftHandThumb1 LeftHandThumb2 LeftHandThumb3 Neck Head RightShoulder RightArm RightForeArm RightHand RightHandIndex1 RightHandIndex2 RightHandIndex3 RightHandMiddle1 RightHandMiddle2 RightHandMiddle3 RightHandThumb1 RightHandThumb2 RightHandThumb3 joint8 joint9 leftpad rightpad sword -scale .92

* md5mesh

 

*md5anims (no need to add the source maya binaries or destination names since they are already in tdm_humanoid_animations export decl. & they'll use the -sourcedir of the main def)

addoptions -inherit tdm_humanoid_animations -destdir models/md5/chars/inventors/engineer -scale .92

 

I'll post more once I gets my thoughts of the matter more organized. Also we could do the same for the animation decls. More on that once I get those thoughts organized.

 

EDIT: we may need to find the author of the maya 7 plugin so we can get the source. Doom3World is the place to look but that thread is extremely hard to find.

[ascottk]

Okay, I just had a thought on animations decls. There is a model decl in the def files:

model tdm_ai_citywatch {
mesh					   models/md5/chars/guards/citywatch/tdm_ai_citywatch.md5mesh

channel torso		( *Spine1 )
channel legs		( *origin - *Spine )

anim af_pose		 models/md5/chars/guards/citywatch/af_pose.md5anim
anim ik_pose		 models/md5/chars/guards/citywatch/af_pose.md5anim
anim walk			  models/md5/chars/guards/citywatch/walk.md5anim {
   frame 11	footstep
   frame 28	footstep
}
anim walk_torch			  models/md5/chars/guards/citywatch/walk_torch.md5anim {
   frame 11	footstep
   frame 28	footstep
}
anim walk_blind			  models/md5/chars/guards/citywatch/blindwalk.md5anim {
   frame 10	footstep
   frame 29	footstep
}
	etc.
}

The problem with this system is you can't separate the animations from the models declaration. Suppose we were to do something like this:

model tdm_ai_citywatch {
mesh					   models/md5/chars/guards/citywatch/tdm_ai_citywatch.md5mesh

channel torso		( *Spine1 )
channel legs		( *origin - *Spine )
}

animations tdm_citywatch_anim {
anim af_pose		 models/md5/chars/guards/citywatch/af_pose.md5anim
anim ik_pose		 models/md5/chars/guards/citywatch/af_pose.md5anim
anim walk			  models/md5/chars/guards/citywatch/walk.md5anim {
   frame 11	footstep
   frame 28	footstep
}
anim walk_torch			  models/md5/chars/guards/citywatch/walk_torch.md5anim {
   frame 11	footstep
   frame 28	footstep
}
anim walk_blind			  models/md5/chars/guards/citywatch/blindwalk.md5anim {
   frame 10	footstep
   frame 29	footstep
}
	etc.
}

 

Then for the engineer, we'd do something like this:

model tdm_ai_inventor_engineer {
mesh					   models/md5/chars/inventors/engineer/tdm_ai_inventor_engineer.md5mesh

channel torso		( *Spine1 )
channel legs		( *origin - *Spine )
}

animations tdm_inventor_engineer_anim {
inherit  tdm_citywatch_anim
dir models/md5/chars/inventors/engineer
}

That way we'd use the citywatch def file for the animation decls but using the md5anim in the engineer's directory.

[sparhawk]

I don't know how the animations def file looks, but if they always look simialr, maybe one of us can write you a small app, the applies an animation to all the AI?

[Domarius]

That way we'd use the citywatch def file for the animation decls but using the md5anim in the engineer's directory.

So what you're saying in that post is you can just use def file inheritance after all?

[ascottk]

So what you're saying in that post is you can just use def file inheritance after all?

That'd be my preference but I don't think we can yet.

[Domarius]

Yeah Crispy said that def parsing isn't in the provided SDK.

 

I think plan B could be what I suggested - write the def file as you demonstrated up there, and have the programmers make a little utility that parses the def file and creates new def files, replacing the "inherit" line with the stuff it's actually inheriting.