Some more debugging using actual debug tools on 2.12, and I've also had a better look at the history and compared the two offending versions better.
The major change between version 2.06 and 2.07 is that the lightgem was moved to a separate subview (or viewport) for rendering. Most of the calculation logic was moved from LightGem.cpp to tr_subview.cpp and it was also adapted to the new system.
I think I've managed to narrow it down to two core issues:
1. Positioning
From what I gather, the gem position should not be adjusted or calculated in tr_subview.cpp. If the subview is only used for rendering the light gem, I presume it should not also have to deal with the logic of its positioning relative to the player/camera. Also, if that's the case, that makes it redundant, since the lightgem's position is already adjusted in LightGem.cpp. To solve this, I simply commented out the position adjustment code in tr_subview.cpp(lines 550-564), and also added the missing lg->SetOrigin( LGPos ) in LightGem.cpp. If this logic is sound, it's presumably a copy-paste artifact from when the code was being moved and adapted to subview.
2. Shadows in the light
My current theory is that the player, one of its attachments, or something else entirely is casting a shadow in the VID_LIGHTGEM render layer. Now obviously, the player and the head are specifically flagged not to render their models and shadows in the subview:
The only question then is what is actually casting that shadow.
In the attached video you can see that this theory holds water. Since the gem only moves a little bit towards the lean position, it makes sense that the player leaning towards the light source would obscure the light and darken it. Turning away from the light makes it brighter.
As a side note, player shadow being enabled/disabled has no effect on this.
Sullium mentioned above that it seems to have something to do with light elevation, which is kind of true. Since the player is leaning and offsetting on a flat plane, an overhead light will exhibit this problem less than one at eye level.
Now, this issue is 90% mitigated by fully moving the gem position to the camera position:
However, that's not the real solution. It makes it so that when the player walks around it makes the light level very responsive, but also very jittery. I suppose the player walking/running animations are casting a shadow on the light gem when moving. It's a bit of a leap, but from what I can see it seems consistent. The light level also jitters more slowly when the player is crouching.
I have no solutions for this, as I'm not yet familiar enough with the rendering system, or perhaps any rogue attachments on the player/camera.
EDIT: Scratch that, I think I got it.
// Added this to tr_subview.cpp:567
lightgemRv.vieworg = lg->GetPhysics()->GetOrigin();
Unpacked a fresh copy of the 2.12 source, added this line in, seems to work like a charm. I was considering that the lightgem camera might be offset from where it's actually supposed to be and then I saw that the vieworg is not set anywhere in the subview, when the view should be at the center of the gem model. It just works™.
I'll remove the position adjustment in subview as stated in point 1 above, since it really does seem redundant, and test these changes on a few missions. I'll also test it out with func_peek, since I noticed an issue mentioning it.