Idea: Demo support, watch recordings of playthroughs as a floating camera / player 1st person / 1st person view from AI

[MirceaKitsune]

As usual this is mainly intended as a discussion and to hear opinions: It's not an expectation that the feature will happen soon or at all. Especially as this would be complicated to achieve, but if it could be done I think it may be a fun capability to consider.

The idTech engine had several features that got removed since we aren't using them like multiplayer. One used to be the ability to record a demo of your matches, which makes sense in multiplayer DeathMatch games. Yet recently I started to wonder: How would it be like to have a demo capability designed for TDM?

Imagine being able to record your playthroughs or share your recordings for others to watch, for when you'd like to relax watching a FM like a film instead of playing it like a game. You could fly as a detached camera, observing yourself wonder around the map and watching the things you did... or stay in 1st person and see the game play itself like in a Youtube video. The coolest thing would be the ability to see through the eyes of any NPC: Imagine being able to click on a guard in floating camera mode to embody them, seeing through their eyes as they patrol the area and at some point notice the player hiding in a corner through that guard's eyes (even if the AI didn't notice you).

The biggest challenge would be a way to record store and reproduce every input from the player, with every AI decision and other entity actions just as they occurred, which includes preserving object physics to represent the exact movements of all entities: Parts of the demo system from Quake would likely need to be reimplemented, wonder how much it knows to handle on its own. Saves and loads would be another tricky one: Each demo should erase what you did after last loading to produce a seamless run... alternatively it could record that as an action and include reloading in the playback. There's probably other challenges but if it's within the realm of possibility, would anyone else use this and think it's worth thinking of?

[jaxa]

I think there's randomness in parts of the game, like NPC reactions. It probably won't work.

[MirceaKitsune]

25 minutes ago, jaxa said:

I think there's randomness in parts of the game, like NPC reactions. It probably won't work.

Those would need to be recorded and made constant during playback of course. Physics aren't random though right... like if you pick up and throw a vase against a wall, it will always move and roll the same way based on initial velocity? If so I'd presume this could work by merely remembering and simulating identical player key-presses and mouse movements, while using constant values for randoms in AI decision making and other entity scripts to ensure they always do the same things.

[Daft Mugi]

@MirceaKitsune Have you tried the following?

recordDemo <name>
stopRecording
playDemo <name>

It does work to a degree, but I don't know how extensive it is or if there are bugs. Plus, FPS drops considerably -- at least on my machine.

[MirceaKitsune]

1 minute ago, Daft Mugi said:

@MirceaKitsune Have you tried the following?

recordDemo <name>
stopRecording
playDemo <name>

It does work to a degree, but I don't know how extensive it is or if there are bugs. Plus, FPS drops considerably -- at least on my machine.

Oh my, it's already there? I will definitely take a look at it at some point! If it works there should definitely be a menu component so it's accessible to everyone, I didn't even hope it was already implemented to some extent

Will need to see how it handles saving and loading while a demo is being recorded. Also what camera modes are supported: The magic I had in mind would be the ability to fly around and see the player, as well as through AI's eyes to experience how they perceived the world and saw you during your run.

[Daft Mugi]

4 minutes ago, MirceaKitsune said:

Oh my, it's already there?

Yeah, it was originally part of the Doom 3 engine.

[MirceaKitsune]

2 hours ago, Daft Mugi said:

Yeah, it was originally part of the Doom 3 engine.

Indeed, but I imagined it was removed with the other stuff... now I hope it won't be, and if need be can be fixed to be compatible with TDM. The question is if it knows to work with our AI and entities, if we haven't done anything too radical to break it maybe we're lucky and it does! I should finish the FM I'm currently on, then I'll reload a previous one and give it a quick try to at least ensure nothing crashes or goes crazy.

Edited October 30, 2023 by MirceaKitsune

[stgatilov]

There are two recordings in the engine.

One is "avi" or "render demo" recording, which records the data which flows into the renderer. I generates a lot of data, but I think it worked more or less recently.

The second kind records player inputs, and replays them later. Indeed, randomness has to be deterministic to make this work. However, it does not work: the player movements don't repeat. I tried to fix it, but could not find the problem.

[boissiere]

So which type of recording is done by the "recordDemo" command?

I just tried it, following Daft Mugi's instructions and it worked reasonably well but

• The replay had no sound
• I had some entity guis (signs) in my test map and the text wasn't shown on the replay - the signs were just blank
• The saved file was nearly 2MB for ~20 secs of recording (and wasn't particularly compressable either)

I'm guessing (from the file size) that this was an 'avi' recording.

[freyk]

A list with recording commands and the descriptions, can be found at:

https://tweakguides.pcgamingwiki.com/Doom3_9.html

[MirceaKitsune]

Apologies for how ridiculously long I took to get to testing this. I was waiting for a short FM I could experiment with to try out demo recording. Now that I've come across Eye on the Prize I figured I'd see what a short demo behaves like.

So what is the verdict? I'd say almost functional! Close but still far yet not as much as I imagined. There are some very obvious issues that currently make the system unusable in practice, but I was glad that it seems to work at the core basics. At least during a simple test where I went around picking up a body and some loot for about 10 seconds, a longer playthrough may reveal more. I'll proceed to list all problems I observed big and small:

1. To get the mundane out of the way, I noticed that unlike screenshots demos are recorded in the FM directory rather than the root directory. While a screenshot is saved to "darkmod/screenshots" a demo is instead stored in "darkmod/fms/active_fm/demos". This is the least concern and easiest to solve, the real problems come next.
2. Demo recording greatly reduces FPS, to the point where it isn't practical to leave it running. I went from 100 FPS to 30 FPS while it was running, constantly jittering as it would speed up a few frames then slow down again. The performance impact would need to be reduced for it to be usable during a normal run.
3. The file size of the resulting demo is uncomfortably large, which isn't a bug per say but a blocker to practical usage. I got a 7.5 MB file for just 15 seconds on an extremely tiny FM: If someone were to record one hour on a large map with tons of AI and entities, I fear it would well surpass 10 GB if we're lucky.
4. Playback, audio: Now for the playback issues, the first of which is lack of audio. The demo has no sound at all, neither world audio nor footsteps or the sounds for picking up loot.
5. Playback, player model: While the camera appears to remain in 1st person, the 3rd person player model is still rendered on top. This causes you to see polygons from the player's head covering the screen.
6. Playback, visual glitches: Lastly there is a strange issue with some parts of the world appearing grey or black. As you move around you'll see geometry become a dark color as if rendering is broken.

Here's a screenshot of demo playback showing issues #4 and #5. The thing you see covering the view at the left is the player's head seen from the inside as it was bobbing around. In front is a normal structure which for some strange reason has its edge appearing black, the floor itself appears gray at some point which I didn't capture here.

Which of those problems if any seem fixable? I presume whatever causes the player model to show and the world to glitch can be found then audio restored, but can performance and storage size be improved? Are the issues simple enough so that it's worth the effort trying to tackle them and see what we get working in the end?

Edited August 1, 2024 by MirceaKitsune

[boissiere]

Hard disk performance might have a bearing on point 2 as it's continually writing the render info to the disk.

For point 3, I don't think the overall size of the FM particularly matters just 'busyness of scene'.

[MirceaKitsune]

16 minutes ago, boissiere said:

Hard disk performance might have a bearing on point 2 as it's continually writing the render info to the disk.

For point 3, I don't think the overall size of the FM particularly matters just 'busyness of scene'.

That sounds highly plausible. If disk performance is the bottleneck, the obvious solution would be to write the demo in memory and only save to the drive once you stop recording. Though with such large demo sizes that could cause some players memory to fill up: Dump it in steps instead like say every 100 MB or 1 GB, or do so every 1 minute if using a timer is better.

Indeed it's not the scale of the map that matters but the number of entities, large maps tend to have more objects so usually one follows the other. The solution here would be to only record entities that changed: If a 7.5 MB demo was generated just from me moving around and picking up loot for 15 seconds, it was likely not recording just my movements but the state of every entity per frame which is extremely wasteful if that's the case.