I verified that the problem is when the AAS file is created, not when the AI is moving at game time.
So the "maxs" height value of "82" defined in aas48 is being used by dmap when it creates the *.aas48 file.
For humanoids, the "maxs" height is "68", which matches the "size" vertical definition of "68" for humanoids.
I examined the doom3 def files, and found numerous monsters using aas48, and their heights were in the "82" range. So aas48 was originally designed for creatures taller than a human, but wider.
So I'm guessing that when these monsters all went bye bye in the early stages of TDM, and the large spider was created, the designer(s) simply dropped him into the now unused aas48 category, w/o moving the height value down from 82 to around 32.
Our only other monster who uses aas48 is the werebeast, and since he doesn't exist yet, we can ignore him in any future changes we make. If he ever shows up, we can give him his own aas designation if that's the right thing to do.
So what happens now?
Two paths, as I see it:
1. Change aas48's height value down from 82 to 32 (or so) in the aas.def file. Since the large spider is the only creature using this, we only need to retest existing maps that use the large spider. By a quick count, this is roughly 6 maps. (More digging might reveal a few more from the latest crop of maps.)
a - Leave the affected maps as they are. They'll retain their current behavior, because they were built with versions of dmap where the height value used was 82.
b1 - Re-dmap the affected maps using the new aas48 definition, creating new *.aas48 files for them. Don't change the *.map files. Huge spiders would begin walking under lower ceilings, changing the dynamics of the map.
b2 - Re-dmap the affected maps using the new aas48 definition, creating new *.aas48 files for them. Change the *.map files with the author's permission to either retain the original spider behavior, or take advantage of the new behavior.
2. Create a new aasNN definition with the proper height for huge spiders, and give that to the spiders dropped into maps that would depend on 2.07+.
The same approach would be taken for the lantern bot, which has an incorrect aas96 height definition.
In the case of the lantern bot, he shares aas96 with a not-yet-created humanoid-size guard bot, so this guard bot falls into the same category as the werebeast: not here yet, so ignore him for now.