Visportals vs func_portals vs antiportals

[SteveL]

Moved from the VP discussion in the Invisible Brush of Death thread:

 

While you are looking at portal changes, any chance you could look into implementing "antiportals" http://udn.epicgames...ntiportals.html Eg. Invisible brush volumes that tell the renderer "do not render anything behind me when in view". I suppose func_portals serve this same purpose but have more overhead (not as low level). (an old Doom 3 mapper complaining that antiportals don't exist in Doom 3: http://www.celephais...13826&end=13850 )

 

Interesting links as always nb. I’d not heard of antiportals before and I’ve only been studying the code since Monday but here’s my twopenceworth anyway.

 

I think you’d need to keep track of a separate system of occluded frustums to go with the current system of view frustums. You can’t just slice parts out of the current frustums without ending up with complicated concave 3d shapes. Right now a view frustum is good for one whole visleaf, and the cross-section is always a convex polygon, which makes the number crunching easy. So you’d want a separate list of anti-view frustums, and surfaces would get checked multiple times: once for each current view frustum for that visleaf (you only get >1 where >1 open portals look on a visleaf) and once against each “anti” frustum generated by an antiportal.

 

Time to play devil’s advocate though: why would we need them? I reckon the examples in those posts could be handled well by the existing system.

 

In the first link, there's a boxed-in room and a long hill that hides a forest behind it. The box room is a non-issue for us, there's no need to do anything special. Just make sure the walls seal and plug the door with a VP. The hill is more interesting, but the normal portalling system still applies, just on a bigger scale.. Put a caulk wall inside your hill and top it with a VP that extends up to the skybox. So the forest ends up in a different visleaf. That VP will always be open when the player looks in that direction, but the forest won't be in the view frustum when the player looks through it from ground level. They'll just see skybox.

 

Now the forest will get checked to see whether it's in the view frustum, but I can't see that antiportals would help with that. The forest would need checking anyway to see whether it's in the occluded zone.

 

In the second article, the counter blocks the view between two doorways. Again, it can have a caulk wall in it and a VP above and to the side to cut the room into 2 visleafs. The poster gives that solution himself but mentions it as a pain. I'm not sure why though: it's an efficient solution using just 3 brushes and it's easy to do.

 

One interesting thing I've learned while looking into this bug: a VP doesn't need to be closed to give a lot of benefit. A closed VP means the area behind it doesn’t need checking at all. An open VP means what’s behind it gets checked to see if it’s in line of sight, but the VP will still prevent anything that isn’t in LOS from sending any light/surface interactions to the back-end renderer. Here’s what the above example looks like with the central VP missing:

 

 

With the VP, a single slice of the visleaf behind portal 2 generates interactions. Without it, the whole room does, and it’s left to the back-end with its more expensive, more precise algorithms to decide what’s really in view and what isn’t.

[Tels]

Time to play devil’s advocate though: why would we need them? I reckon the examples in those posts could be handled well by the existing system.

 

In the first link, there's a boxed-in room and a long hill that hides a forest behind it. The box room is a non-issue for us, there's no need to do anything special. Just make sure the walls seal and plug the door with a VP.

 

Just a quick reply to this point alone:

 

As far as I understand their example, the box-room has an open door, and is just used as an example where the wall (and hill) should block things, but don't.

 

I think we have the same situation in "house on street level has two corners visible and an open door". e.g. you can see inside the house, and are outside. In this case you can see only a small part of the inside of the house (the sliver through the open door), but the entire interiour gets rendered, because the wall does not oclude.

 

(if the other rooms have doors, and these are not in line with the front door, then these rooms get blocked.)

 

In any event, I think it has its places, the question is, is it worth the added complexity in the engine and will many mappers use it?

 

I certainly have a few things in mind where I really could use this technique (wide open area with enclosed spaces in it, like a market place, a desert, or a huge underground cavern). Everytime you stand next to a wall etc. you still see the entire open space, unless you are in an enclosed space with a visportal. But in some cases you cannot add portals (like if you are in an half-open destroyed house), while in other cases you can (normal house).

[Tels]

Another example is something like a huge statue in an open space - if you are right next to the statue and look at it and have a window behind it, then the statue does not occlude, so TDM sees the open window and renders the interiour of the house, even tho you can't see it. In this case it might be very hard to devide the marketplace with visportals. Putting in an ocluder and/or anti-portal in the statue seems easier.

 

An ocluder would occlude geometry (so you would not "see" the Ai behind the statue, but the window would still be an open VP), and an anti-portal would just make sure that whenever another poral is completely inside its frustrum, the other portal gets closed (so you see the AI behind the statue, but the window is closed).

 

Was my description clear? I could draw a picture

[nbohr1more]

Yeah, the more I look into it, the more antiportals look like a nice feature for outdoor scenarios since they forcibly occlude in a more precise way, even more precise than func_portals as I can tell. It seems Id Tech 4 is the only portal based engine without them. I tried to find a code example at github but it seems that Quake 3 was tricked into using the via a new map compiler so the code there would not be quite comparable.

[Tels]

Technically, an anti-portal would just work entirely like a normal portal, but the logic when to expand the visible space would be reversed. Kinda like:

 

* normal portal: If THIS portal is open and NEXT portal is completely inside my frustrum, and open, expand the visible space

* anti-portal: (IGNORE state of THIS portal, never open or close it) If NEXT portal is completely inside my frustrum, and open, do not expand the visible space (ignore all other cases, treat portal like it was "open" or "not there)

 

Might be that having a flag (and new code logic)on the portal is enough. In any event, it probably needs a new texture so you can construct it in DR. Also, an anit-portal would not need to split the map and all that, it would just be a plane on a block. Not sure how easy this is to implement.

 

Ocluders are bit more complicated, and a separate case, and we might do without them for now.

[SteveL]

I'm going to think some more about the statue example. I was all ready with a reply re roofless houses and marketplaces (which I still think can be done more efficiently with the existing system than with an antiportal), but I have no ready answer to the statue. And I guess there must be some reason that other engines use them

 

One point of order: I notice we mean different things by "gets rendered". In my post up top, I use it to mean gets passed by the front end to the back end to be drawn, whereas your house examples use it to mean goes into the front end. It's not a yes/no question whether something gets rendered of course. The job of the front end is to cut down the work for the back end by deciding what needs to be drawn and what doesn't. Different things can drop out at different stages of that process, with a different amount of work having been done.

 

My point is that an open visportal sitting on a block of caulk does the same work that an antiportal would. Take your broken-open house for example. A bit you'd want to occlude would be the floor of a room when the player is the other side of a broken wall and can't see it. If you put a VP on top of that wall leading all the way up to the skybox, maybe with a caulk insert to the wall if the top is crumbly, you get the same effect as if you had an occluder in the wall. Completely missing wall? Use a vp from floor to ceiling. It won't hide anything, but no problem, you just need to finish the job of sealing the area.

 

In both cases (caulk+vp vs antiportal/occluder) the rendering work for the unseen stuff behind will end at the same stage, when surfaces are tested to see whether they lie in a certain frustum or not.

 

However, occluding behind a statue... requires the occlusion to happen from any angle and I can't think of an answer to that using VPs. Hmmm

[Tels]

I'm going to think some more about the statue example. I was all ready with a reply re roofless houses and marketplaces (which I still think can be done more efficiently with the existing system than with an antiportal), but I have no ready answer to the statue. And I guess there must be some reason that other engines use them

 

One point of order: I notice we mean different things by "gets rendered". In my post up top, I use it to mean gets passed by the front end to the back end to be drawn, whereas your house examples use it to mean goes into the front end. It's not a yes/no question whether something gets rendered of course. The job of the front end is to cut down the work for the back end by deciding what needs to be drawn and what doesn't. Different things can drop out at different stages of that process, with a different amount of work having been done.

 

Yes, sorry, I was simplifying (and also, all these little details left my memory a year ago

 

My point is that an open visportal sitting on a block of caulk does the same work that an antiportal would. Take your broken-open house for example. A bit you'd want to occlude would be the floor of a room when the player is the other side of a broken wall and can't see it. If you put a VP on top of that wall leading all the way up to the skybox, maybe with a caulk insert to the wall if the top is crumbly, you get the same effect as if you had an occluder in the wall. Completely missing wall? Use a vp from floor to ceiling. It won't hide anything, but no problem, you just need to finish the job of sealing the area.

 

Yeah, you can do a lot with visportals. But I think the point was that it is:

 

* sometimes not possible (think very large open space with a ot of "half-way open" structures strewn across. You can't just cut the open space with hundres of visportals across. Also, visportals always divide the world. This will create extra triangles and creates extra work in some places. So the less visportals you use to devide the world, the better. Unless of course it adds to more stuff not getting culled

* and visportaling can be a pain-in-the-nether-regions. The example from the Quake (?) map is exactly this - the anti-portal in the middle column is much easier than cutting the room up with multiple portals just to cover each of the various door-openings.

 

In both cases (caulk+vp vs antiportal/occluder) the rendering work for the unseen stuff behind will end at the same stage, when surfaces are tested to see whether they lie in a certain frustum or not.

 

However, occluding behind a statue... requires the occlusion to happen from any angle and I can't think of an answer to that using VPs. Hmmm

 

I forgot how it is when you add portals at 90° covering the "ceiling". Does that work? You can then cut a place into "below the statue head" and "over the statue" head. But portals that cross into each other at 90° have probably some other problems.

 

As for the "occluder" or "anti-portal in the statue or column, the trick is usually to make a cross-shape "+" where you have two planes at 90° angles. These are, when viewed from any angle, always covering the inside of a "round structure". Of course that does not really work with visportals as they divide the geometry. Anti-portals would not divide the geometry, so they would be safe. The same trick is often used to represend round stuff like pipes, vegetation etc. at a far-aways scale. Two flat planes look the same like a round pipe but with less triangles.

 

Making a billboard plane (like decals), e.g.one that always rotates around to face the player, is another solution. However, visportals do not have that ability (they are static), and also, AFAIR, billboards in idTech 4 have the strange property not being able to be restricted to the z-axis - meaning if the player leans, they lean, too. Which would not work for a round column or statue (and already causes strange artefacts with vegetation build from billboards).

 

After re-reading the posts, I think anti-portals are useful, but not crucial. One can get away without using them, but it is handy to have them in some situations.

 

AVPs would/could/should also partially block sound (by routing around them), but I'm not sure how hard that is to implement.

[Tels]

Another thought: Would an anti-portal not always be an occluder, too?

 

However, if you stick an occluder inside a wall/column on an open space, and the renderer skips then any object occluded by it, then you will miss shadows. How does that work with an AI being in the next room, with the door open, but behind a wall? Even if the AI is not visible, the shadow might be. So I think occluders are even harder to implement than anti-portals.

[SteveL]

Another thought: Would an anti-portal not always be an occluder, too?

 

Every time I read the posts above, I think, "Ah, for the first time I now understand what's meant by an anti-portal". This time I do think I've understood. It's not actually a portal, that is, a division between 2 BSP areas, it's just an abitrary shape or volume that has the power to close any VP seen through it. If I've got the right end of the stick now, then I agree with you, it'd be easy to implement. To put it another way, it's an occluder that applies to whole visleafs only. I now see the connection with a func-portal.

 

However, if you stick an occluder inside a wall/column on an open space, and the renderer skips then any object occluded by it, then you will miss shadows. How does that work with an AI being in the next room, with the door open, but behind a wall? Even if the AI is not visible, the shadow might be. So I think occluders are even harder to implement than anti-portals.

 

I've not looked at the relevant bit of the code, but as you suspect, this problem has already been solved in the renderer front-end. An entity can be outside the view frustum and so generate no interactions, but the AI's shadow can get added later in the process to the interactions passed to the back end. Any shadowcasters in the visleaf are checked separately to see whether their shadows might be visible even if they are not. So it's probably solveable. The place to put an antiportal test would be in the same bit of code I've been dealing with, that decides whether to flow the view through a portal or not. The place to implement an occluder would be in the code that decides whether surfaces are in the view frustum or not. Both those things happen before the mop-up that adds shadow interactions for non-visible entities.

 

Are there portalled engines comparable to TDM's that use occluders and antiportals? I've done a bit of googling and seen that they were in Unreal before version 3, and found a few examples of them being used in pvs engines like Quake, but not found anything really relevant yet.

 

Re func portals

 

They can be used to control the view to an entire visleaf of course, and we already have them. Under the bonnet they're just another VP: if anything makes them a bit less efficient I guess it's probably the repetitive distance checking by the FP entity (although that's surely so cheap that it shouldn't be a concern). The one thing I've thought about that could improve them is the manual control. You can trigger them to open or close them but that's not so useful as being able to command them to close or open, say from location scripts. It's easy to imagine the player doing something like climb over a wall that you hadn't accounted for, making your trigger system get out of sync leaving the FP closed when it should be open or vice versa. Could fixing that help with some of the use cases for an anti portal?

[Tels]

Every time I read the posts above, I think, "Ah, for the first time I now understand what's meant by an anti-portal". This time I do think I've understood. It's not actually a portal, that is, a division between 2 BSP areas, it's just an abitrary shape or volume that has the power to close any VP seen through it. If I've got the right end of the stick now, then I agree with you, it'd be easy to implement. To put it another way, it's an occluder that applies to whole visleafs only. I now see the connection with a func-portal.

 

By god, he got it! :-D

 

(I freely admit I'm crap at describing things Plus, at first I thought that the "anti-visportal" is like a visportal, too, only later when I described what it should do, or not do, it became a bit clearer.

 

I've not looked at the relevant bit of the code, but as you suspect, this problem has already been solved in the renderer front-end. An entity can be outside the view frustum and so generate no interactions, but the AI's shadow can get added later in the process to the interactions passed to the back end. Any shadowcasters in the visleaf are checked separately to see whether their shadows might be visible even if they are not. So it's probably solveable. The place to put an antiportal test would be in the same bit of code I've been dealing with, that decides whether to flow the view through a portal or not. The place to implement an occluder would be in the code that decides whether surfaces are in the view frustum or not. Both those things happen before the mop-up that adds shadow interactions for non-visible entities.

 

Ah, that sounds actually better than I thought.

 

Are there portalled engines comparable to TDM's that use occluders and antiportals? I've done a bit of googling and seen that they were in Unreal before version 3, and found a few examples of them being used in pvs engines like Quake, but not found anything really relevant yet.

 

I never heard of the term before this discussion came up, but in hind-sight the idea sounds one of these "it's so simply, why didn't I think of it!"

 

Re func portals

 

They can be used to control the view to an entire visleaf of course, and we already have them. Under the bonnet they're just another VP: if anything makes them a bit less efficient I guess it's probably the repetitive distance checking by the FP entity (although that's surely so cheap that it shouldn't be a concern). The one thing I've thought about that could improve them is the manual control. You can trigger them to open or close them but that's not so useful as being able to command them to close or open, say from location scripts. It's easy to imagine the player doing something like climb over a wall that you hadn't accounted for, making your trigger system get out of sync leaving the FP closed when it should be open or vice versa. Could fixing that help with some of the use cases for an anti portal?

 

Maybe. But an automatic antiportal sounds much easier to use than the fiddly func portals. They strike me more like a hack than anything else. Which has it's place, of course (like making semi-transparent windows opaque on distance).

[SteveL]

[offtopic] @Tels: I can't PM you? Is that deliberate or do you just need to delete some stuff? Not about this discussion, just cheekily trying to get some advice on extending LOD

[Tels]

Uh, I'll clean up my attic er PM folder, or do whatever is necessary...

 

Edit: Ah,yes, for some strange reason I can only have 50 "conversations", but overall can seemingly have an unlimited amount of messages (some of my conversations have nearly 100 messages). Cleaned up old cruft. PM away

Edited August 15, 2014 by Tels

[SteveL]

Intriguing suggestion. You'd only need one frustum per occluder on screen per frame, and it'd be good for all visleafs, unlike normal view frustums that need reshaping for every visleaf. You'd make it a 2d object like VPs are, so there'd be no need to calculate silhouettes or write new code: instead it would just use the existing code for detecting VPs that are in view, and for shaping the view frustum through that shape.

[SteveL]

Just having a play around to see what the existing VP occlusion does with models. I think I see why the instructions in article #1 above say that all of a model must be covered by the occluder for it to be hidden. Seems you can't take bites out of a model, leaving a concave stencil to be rendered. You can clip it using visportals, because any combo of VPs seen through one another will always result in a convex scissor shape to be rendered, but for an occluder it looks like has to be an all-or-nothing thing.

 

You can see the same thing happening with VP occlusion right now. The object on the right is a barrel seen through 1 VP, so what's left is convex and it gets partly drawn. The AI though is visible through 2 VPs and could have only 1 leg culled, so he's shown in full.

 

And if you move so the barrel (bounds) can be seen through more than 1 vp, again the whole thing gets shown.

 

So you wouldn't get nice neat scissored edges with an occluder, but on the plus side that simplifies the task of trying it out quite a bit.

 

Shadow casting is independent of visible model culling, as hoped. I guess that's why you couldn't try an occluder in the game code. Hiding the model would hide the shadow too, and anyway it wouldn't have done anything to the worldspawn's visible models.

[SteveL]

So do you guys think that whole-entity occlusion is worth testing? I do for what it's worth: my initial scepticism is overcome and I thought of half a dozen places I'd have liked to use it in a map a work on occasionally There's apparently a really obvious way to do it using existing front end renderer techniques, the same ones I was looking into earlier this week, and I'm pretty hopeful that the efficiency (tris culled vs tests done) will be at least as good as portal occlusion. If anyone has any more info on how other games have improved the basic set up, please share!

 

Not sure how similar tdm's worldspawn setup is compared with other games (because I know only tdm's). Dmap takes all visible worldspawn surfaces within a visleaf and turns them into one or more models that get written into the proc file. So typically the inner WS faces of a room would be a single model and not occludable. But I don't see that matters for now, and it's the func statics that have the detail.

[Tels]

I'd like the "close visportal leafs" first (because the anti-portal occluder can close a window behind a wall/statue/whatever, the savings are probably bigger).

 

As for the occlusion of func-statics, well, the check might be cheap (I don't know for sure) and the benefits huge. If it only removes one AI from view, that would pay already.

 

Imagine you are crouched against a free-standing bookcase, and there are very detailed entities hidden behind it (AI, vases, plants, or a detailed desk with many items). Even if it only works for completely hidden items.

 

But ultimatively, it is up to you. I can only arm-chair here

[SteveL]

I'm hoping we won't have to make a choice. VP blocking looks to be trivial, I mean *1* extra line of code implementing one very cheap test. It wouldn't even have to close the VP. All it needs to do is prevent the same routine that was screwing up in the nhat freezes from flowing the view through that portal. And the whole-entity occlusion happens in the same function, so it's all local too.

 

Occluders are not part of the bsp and there's no reason why they can't be created, resized and destroyed at runtime. So a func_occluder could be a lot more flexible than a func_portal. You could bind one to a moving entity, for example.

 

I hope it works as expected after all this speculation on my part! I branched my working copy at 6 am this morning and inserted the code that'll create the anti-view frustums for the frame, using the existing view frustum-generating routine, but I didn't get as far as doing the real proof of concept test before having to leave for the day :-\ I'm excited about testing it. Right now I'm out on a real ale pub crawl with friends, and I'm putting up with ridicule for drinking half measures purely because I want to be able to test later! Although being realistic I should aim for tomorrow *hic*.

[nbohr1more]

Keep it up!

 

All I can say is that I am excited about the upcoming tests!

[Tels]

I'm out on a real ale pub crawl with friends, and I'm putting up with ridicule for drinking half measures purely because I want to be able to test later! Although being realistic I should aim for tomorrow *hic*.

 

Please don't let TDM prevent pub crawls! (Ah, the fond memories

[Paralytik]

I'm also looking forward to seeing this, if it works out well, in action.

[Springheel]

the whole-entity occlusion happens in the same function, so it's all local too.

 

I don't fully follow all of this whole-entity occlusion discussion, but having func_statics display behind portals is a feature used to fake distance in city maps... don't know if that would be affected.

[SteveL]

I don't fully follow all of this, but having func_statics display behind portals is a feature used to fake distance in city maps... That needs to be kept.

 

Thanks, added to the test list. Could that cause problems? At face value the FS rule sounds compatible with what we're discussing. The FS rule: if any part of an FS is in your current visleaf, the whole of it gets drawn and is visible through caulk. But could an occluder screw that up by hiding the only part of an FS in *this* region, thus blanking out the parts you have in other visleafs? No, it couldn't, I think... an occluder will hide an entity only if the entire entity is behind the occluder, ignoring what's in what visleaf. Should be ok... But now you've got me wondering about my screenshots above. That barrel is an FS and I had caulk surrounding the VPs, so why didn't we see the whole barrel in shot #1? I'll try to work that out. Ah, none of the barrel was in my current visleaf, and the FS rule is probably just another effect of the thing I was experimenting with above, i.e. that a model can only get parts culled if what's left is a convex portion. So the caulk+FS rule might well be a special case of the rule we're experimenting with. Exactly the same thing in fact. If a model can be seen through more than one view (or more than one VP, or in more than one visleaf) it can't have parts culled.

[SteveL]

I don't fully follow all of this

Not sure whether you mean you don't get the idea, or you just don't follow all my confused earlier posts up top (and who would?). But I want to have another go at explaining for anyone wondering what's being proposed, just because it took me a whole day and several protesting posts before I got my head round it. My attempt is probably a bit ill-advised given the excellence of this afternoon's beer, but here goes.

 

Right now we only have one method of restricting which bits of a big map get sent to the graphics card to be drawn each frame: visportals. Whether we add occluders to the game or not, visportals will remain as important as they are now: the backbone of all maps. They define the space, sound propagation, pathing, and most other things. Occluders would be an addition that does a lot less but is much easier to use. Basically they would be 2D shapes that mappers can place in a map (and that can be controlled by scripts, triggers etc, or be bound to moving entities) that simply act like a blackened pane of glass, preventing anything behind them from being sent to the graphics card. "Behind them" means as seen from the current viewpoint, whether that's the player's eyes or the info_skyportal or one of Obsttorte's security cameras. Normal worldspawn doesn't do that. Without a VP to do the work, even if you can't see something from behind a solid wall, the models behind it get sent to the graphics card, and the light/surface interactions get calculated and sent to the back-end renderer, which is what finally decides whether they really need to be drawn after it renders the final scene geometry using precise but slow algorithms. Occluders would be an extra performance optimization tool available to mappers, and one that could be very flexible and easy/intuitive to use.*

 

* always assuming that they work as hoped for of course

[nbohr1more]

I think Springheel may be referring to portalsky tricks? which, if this works properly, the occluder should not occlude either the contents of the portalsky or the portalsky itself...

 

 

[grayman]

Spring isn't referring to the portalsky tricks. Distant architecture is independent of what the portalsky is painting.

 

As long as the engine continues to render all of a func_static sitting behind caulk, if some part of it is in the current PVS (not necessarily visible to the player), we'll be fine.

 

If the changes cause the painting of func_statics to no longer follow this rule, then we'd have two problems: we'd break (some) existing maps and take an important street-view technique away from mappers.