Announcement: SEED system

[AluminumHaste]

I followed these instructions exactly:

 

• In DarkRadiant, create a brush the size of the area you want to cover
• With that brush still selected, and select Create entity and select Darkmod/Info/atdm:seed

Note:

If you want to resize the SEED entity after creating it, select it with SHIFT + and press TAB to select the brush, then resize it normally.

• Now either add the appropriate templates via spawnargs on the SEED, or create the entities that you want to appear and then link them from the SEED. (These entities are sometimes called "targets (because the SEED targets them) or templates (because they are just a blueprint for other entities that should be created). What types of entities can you use?
  • Place any entity by with -> 'Create Entity'. Entities that use the LOD system should be preferred.
  • Draw brushes/patches in DR, then turn them into a func_static.
  • You can also place a model in DR, then use/link it as entity.

  [*] Link each entity from the SEED entity by selecting first the SEED, then the entity, and then press CTRL+K.[*] Put the entities at the height you want them to appear. Alternatively, use the spawnarg "seed_floor" on the entity (see below for more spawnargs). The X/Y or rotation position of the entity does not matter, but you can f.i. use different skins.[*] Remember to dmap whenever you add, delete or change the position/size of a SEED!

Doom 3 just crashes when trying to run the map. Dmapping is fine, but when I click to start the map CPU usage goes up and then Doom3 crashes.

 

EDIT: Took out the douchebag parts. Sorry

[Springheel]

Wow what a wonderful feature. I just followed these instructions exactly:

...

Annnnnnnnnd Doom 3 just crashes when trying to run the map.

 

So were you being sarcastic? I can't tell.

[AluminumHaste]

I'm sorry I'm just frustrated, I stopped months ago trying to get this to work now to run into this problem is incredibly annoying. Yes it does crash I wasn't being sarcastic.

[Sotha]

I'm sorry I'm just frustrated, I stopped months ago trying to get this to work now to run into this problem is incredibly annoying. Yes it does crash I wasn't being sarcastic.

 

Check posts #208-#212. I had a crash too and the cure was found. Maybe the wiki instructions are easily misinterpreted, if we both made the same mistake?

[Tels]

Check posts #208-#212. I had a crash too and the cure was found. Maybe the wiki instructions are easily misinterpreted, if we both made the same mistake?

 

Starting here:

 

http://forums.thedarkmod.com/topic/12107-announcement-seed-system/page__view__findpost__p__255207

 

I am not sure why it crashes, and what the solution is, even after reading the posts above.

 

@AluminumHaste: It shouldn't crash, but hacking D3 is fickly, because there are a lot of hard-coded constants and unknown limits, plus the engine isn't actually supposed to do what the SEED thing does. Sorry for the inconvience. Have some plans for a rewrite, and the open source thing will definitely help, too, but this is in the near future. We could definitely need some coder help.

[Sotha]

I am not sure why it crashes, and what the solution is, even after reading the posts above.

 

 

 

What I did was:

• I removed all the seed_material_grass and seed_probability entries from the SEED entity. I put them into the seed func_statics that target the SEED entity.
• Since the seed_density spawnarg defaults 1.0 I didn't give this spawnarg to the func_statics
• I removed the max_entities spawnarg from the SEED entity.

Now the game crashes upon map load. Always. If I put the max_entities back to the SEED entity, the map starts properly. (NOTE: I've always dmapped between map build changes.)

 

EDIT:

Now I got it to work:

• I removed the max_entities spawnarg from the SEED entity
• I gave all the targetting seed func_statics the spawnarg seed_max_entities 20

No crash, versatile seed models, no models on tombstones. Perfect result.

 

Looks like having a max_entities -spawnarg on the SEED entity causes the crashes? So all the controlling spawnargs are on the FUNC_STATICS which the SEED targets. The SEED itself, in my case, didn't have seed spawnargs.

[Springheel]

I am not sure why it crashes, and what the solution is,

 

I can't help noticing the other day you seemed rather annoyed that Arcturus didn't use SEED in his grass map. Yet here you seem to be admitting that it isn't fully working yet, and could cause unknown map crashes. Isn't that a significant barrier to people actually wanting to use it?

[ungoliant]

I can't help noticing the other day you seemed rather annoyed that Arcturus didn't use SEED in his grass map. Yet here you seem to be admitting that it isn't fully working yet, and could cause unknown map crashes. Isn't that a significant barrier to people actually wanting to use it?

 

chances are nothing will work without testers, so people need to use it for kinks to get worked out. Also a working example of seed in something as epic as that grassmap would likely increase mapper interest 1000 fold. Fruits of labor are meant to be enjoyed by seeing them appreciated and used, with their value recognized and respected. Not that nobody respects or recognizes the seed system, but a little more on the 'used and appreciated' side couldn't hurt.

 

edit: and you should damn well know all that already being a mapper / team member. why you gotta play devil's advocate all the time?

Edited August 12, 2011 by ungoliant

[Tels]

I can't help noticing the other day you seemed rather annoyed that Arcturus didn't use SEED in his grass map. Yet here you seem to be admitting that it isn't fully working yet, and could cause unknown map crashes. Isn't that a significant barrier to people actually wanting to use it?

 

If we'd never use anything that did not crash, or not work 100%, we should stop using D3 altogether and maybe tell the mappers to go build for Unity or something. Large parts of the engine are quite hackish, not to mention the things we build on top of it.

 

Basically, a bit of a chicken-hen problem, if you ask me.

[AluminumHaste]

chances are nothing will work without testers, so people need to use it for kinks to get worked out. Also a working example of seed in something as epic as that grassmap would likely increase mapper interest 1000 fold. Fruits of labor are meant to be enjoyed by seeing them appreciated and used, with their value recognized and respected. Not that nobody respects or recognizes the seed system, but a little more on the 'used and appreciated' side couldn't hurt.

 

 

That's exactly what I'm trying to do, the initial instructions are incredibly brief and easy to follow (a good thing), so maybe once I figure this out I can modify them to prevent these types of crashes.

[AluminumHaste]

Okay it does work, it was crashing because of the entity I was using. I tried using:

 

atdm:nature_plant_large_bigleaf_01

 

And I also tried

 

atdm:nature_plant_large_bigleaf_01_nohide

 

Both of these entities cause Doom3 to crash when targeted by a SEED entity. Using just atdm:nature_heath_calluna worked just fine.

Is there something specific about the original entity that would cause Doom3 to crash?

Okay I included both of those entities just as separate units in the map and Doom3 ran fine.

 

I guess there is something special about these entities as they have shadow maps so I'm guessing they are rudimentary AI?

[demagogue]

In any event, that should be added into the wiki in a troubleshooting section: If it's crashing, try changing the entity you're trying to seed. Some entities can cause D3 to crash, but other similar entities may work.

[AluminumHaste]

Okay so I tried scaling this up to a very large map (16000x16000) and I created the seed entity and created the other entities and everything worked without error. The only issue now is the console shows this:

 

SEED atdm_seed_1: Preparing entities.
SEED atdm_seed_1: Adding class atdm:nature_pine_30.
SEED atdm_seed_1: Adding class atdm:nature_pine.
SEED atdm_seed_1: Adding class atdm:nature_pine_20.
SEED atdm_seed_1: Adding class atdm:nature_tree_01.
SEED atdm_seed_1: Entity count: 27204.
SEED atdm_seed_1: Creating 0 watch list entities took 0 ms.
SEED atdm_seed_1: Creating 512 entities of class atdm:nature_pine (#1 index 0, seed 107238547).
SEED atdm_seed_1: Creating 11831 entities of class atdm:nature_tree_01 (#3 index 1, seed 15283401).
SEED atdm_seed_1: Creating 4685 entities of class atdm:nature_pine_30 (#0 index 2, seed 65729922).
SEED atdm_seed_1: Creating 10176 entities of class atdm:nature_pine_20 (#2 index 3, seed 120519888).
SEED atdm_seed_1: Preparing 0 entities took 13830 ms.
SEED atdm_seed_1: Combined 0 entities into 0 entities, took 0 ms.
SEED atdm_seed_1: Have no entities to control, becoming inactive.
]condump seed_noentities
Dumped console text to seed_noentities.txt.

 

I can see that it creates all the entities just fine, but then doesn't use any of them. I might know why this is happening. Here's a screenshot of the map.

 

As you can see the SEED entity spans the entire height of the mountainous terrain. I've set the FLOOR option on the SEED entity along with sink_min and sink_max and that's all. Now the way I understand how these spawnargs work, is that the entities will be spawned then dropped down until they hit something, then sink down below the surface they touche randomly between 2 and 5 units (as set in sink_min/max).

 

EDIT: Okay since that screenshot was taken I moved the targeted entities up above the terrain to avoid them spawning too low, but that didn't help same issue.

[demagogue]

I see 4 trees there set at the bottom of the screenshot. Are they being floored on the brush? Maybe it doesn't floor on patches but only brushes.

 

Edit: oops! That's the source trees! Nevermind.

I'd play around with the size & placement of the SEED coverage though. Maybe it works best up to a certain size and you have to use 4 of them, or try covering half the mountain height to see if it works on at least that. Or start small on something you know works and gradually work up to a bigger & bigger scale... etc. And try putting a brush somewhere in there, just in case it just likes brushes. I'd just start with something I know works and then change one piece at a time to see where the problem comes in.

[Tels]

Okay it does work, it was crashing because of the entity I was using. I tried using:

 

atdm:nature_plant_large_bigleaf_01

 

And I also tried

 

atdm:nature_plant_large_bigleaf_01_nohide

 

Both of these entities cause Doom3 to crash when targeted by a SEED entity. Using just atdm:nature_heath_calluna worked just fine.

Is there something specific about the original entity that would cause Doom3 to crash?

Okay I included both of those entities just as separate units in the map and Doom3 ran fine.

 

I guess there is something special about these entities as they have shadow maps so I'm guessing they are rudimentary AI?

 

AHA! Well, it is of course bad that it crashes, but its good to know that these specific entities cause it. Theoretically, it should Just Work[tm].

 

Please file a bug report, I will look into it. Bonus points if you make a small testmap.

 

I'll reply to your other post in a minute. Sorry for the problems

 

Edit: Just a quick question: did you also try these shadow casting entities in the big map? I think that maybe there is another hidden limit on how many shadow casting things D3 can have?

 

Sorry, but I never tried it with maps this big as my PC isn't really equipped for this.

[Tels]

Okay so I tried scaling this up to a very large map (16000x16000) and I created the seed entity and created the other entities and everything worked without error. The only issue now is the console shows this:

 

SEED atdm_seed_1: Preparing entities.
SEED atdm_seed_1: Adding class atdm:nature_pine_30.
SEED atdm_seed_1: Adding class atdm:nature_pine.
SEED atdm_seed_1: Adding class atdm:nature_pine_20.
SEED atdm_seed_1: Adding class atdm:nature_tree_01.
SEED atdm_seed_1: Entity count: 27204.
SEED atdm_seed_1: Creating 0 watch list entities took 0 ms.
SEED atdm_seed_1: Creating 512 entities of class atdm:nature_pine (#1 index 0, seed 107238547).
SEED atdm_seed_1: Creating 11831 entities of class atdm:nature_tree_01 (#3 index 1, seed 15283401).
SEED atdm_seed_1: Creating 4685 entities of class atdm:nature_pine_30 (#0 index 2, seed 65729922).
SEED atdm_seed_1: Creating 10176 entities of class atdm:nature_pine_20 (#2 index 3, seed 120519888).
SEED atdm_seed_1: Preparing 0 entities took 13830 ms.
SEED atdm_seed_1: Combined 0 entities into 0 entities, took 0 ms.
SEED atdm_seed_1: Have no entities to control, becoming inactive.
]condump seed_noentities
Dumped console text to seed_noentities.txt.

 

I can see that it creates all the entities just fine, but then doesn't use any of them. I might know why this is happening. Here's a screenshot of the map.

 

As you can see the SEED entity spans the entire height of the mountainous terrain. I've set the FLOOR option on the SEED entity along with sink_min and sink_max and that's all. Now the way I understand how these spawnargs work, is that the entities will be spawned then dropped down until they hit something, then sink down below the surface they touche randomly between 2 and 5 units (as set in sink_min/max).

 

EDIT: Okay since that screenshot was taken I moved the targeted entities up above the terrain to avoid them spawning too low, but that didn't help same issue.

 

Hm, strange. What says the console if you put "debug" "1" on the SEED entity?

 

Edit: Also could you please post the exact spawnargs on the SEED and the entities it targets?

 

As for patch vs. brush, that should work fine, I have used patches in testmaps just fine.

 

Second edit: Certain things only grow on certain materials. It might be that the material setup for the pine is either wrong, or extremely not favouring the texture you have used. It's hard to spot, but I think it is some "earth" one? Or gravel?

 

   // by default nothing can grow at all:
   "seed_probability"      "0"

   // Things cannot grow on metal, stone, tile etc. with a default of 0,
   // but since so nothing would spawn, modify it for certain materials:
   "seed_material_grass"   "1.0"
   "seed_material_dirt"    "0.8"
   "seed_material_moss"    "1.0"
   "seed_material_sand"    "0.7"
   "seed_material_mud"     "0.7"
}

 

Maybe adding:

 

"seed_material_gravel" "0.7"

 

on the pine entities will help?

 

I freely admit that I build the pine entities when I started with SEED, but later on didn't use them much (as they didn't look good in my view).

[AluminumHaste]

Okay I will try those later, I still have to get some screenshots of SH re-released map.

[AluminumHaste]

Okay changing the ground texture to grass didn't make a difference. Adding debug 1 to the SEED entity brings this up:

 

SEED atdm_seed_1: Preparing entities.
SEED atdm_seed_1: Adding class atdm:nature_pine_30.
SEED atdm_seed_1: Adding class atdm:nature_pine.
SEED atdm_seed_1: Adding class atdm:nature_pine_20.
SEED atdm_seed_1: Adding class atdm:nature_tree_01.
SEED atdm_seed_1: Entity count: 27204.
SEED atdm_seed_1: Creating 0 watch list entities took 0 ms.
SEED atdm_seed_1: Creating 4685 entities of class atdm:nature_pine_30 (#0 index 0, seed 67692590).
SEED atdm_seed_1: Creating 10176 entities of class atdm:nature_pine_20 (#2 index 1, seed 4914524).
WARNING: idClipModel::Handle: clip model 0 on 'player1_head' (55) is not a collision or trace model
SEED atdm_seed_1: Creating 11831 entities of class atdm:nature_tree_01 (#3 index 2, seed 38279269).
SEED atdm_seed_1: Creating 512 entities of class atdm:nature_pine (#1 index 3, seed 1904591).
SEED atdm_seed_1: Preparing 1 entities took 13613 ms.
SEED atdm_seed_1: Combined 1 entities into 1 entities, took 0 ms.
atdm_seed_1: spawned 1, culled 0, existing: 1, visible: 1, overall: 59

[AluminumHaste]

Okay it's working now......I don't know how.......I need to take a break and get some screenshots for SH's mission, so I'll just leave you with this:

Ignore the FPS counter it's wrong, it was around 35 FPS in that shot.

 

[Springheel]

chances are nothing will work without testers, so people need to use it for kinks to get worked out.

edit: and you should damn well know all that already being a mapper / team member. why you gotta play devil's advocate all the time?

 

I was considering using SEED in my next map, but when I see other mappers complaining about it crashing, and the developer who created it saying, "I am not sure why it crashes, and what the solution is," and "Have some plans for a rewrite... but this is in the near future."

 

I took that to mean that it needs to be rewritten so that maps don't crash. In other words, it sounds like a feature that is still in development and testing, not something mappers can reliably use on real maps. I'm not sure how my attempt to clarify that is "playing devil's advocate".

 

Based on what followed after my post, maybe the crashes are rare or already solved, I can't tell. But the very fact that I can't tell is still a problem.

 

So I guess my question is, is SEED in a reliable state that can be used by mappers in real missions, or is it still in a testing/rewriting phase where random problems and crashes are to be expected?

[demagogue]

In this case, it looks like AH braved the murky waters and came up with a fish. It's good to see it turned out well, even on such a big scale. I'm interested in it for the grass & flowers & such.

[AluminumHaste]

I was considering using SEED in my next map, but when I see other mappers complaining about it crashing, and the developer who created it saying, "I am not sure why it crashes, and what the solution is," and "Have some plans for a rewrite... but this is in the near future."

 

I took that to mean that it needs to be rewritten so that maps don't crash. In other words, it sounds like a feature that is still in development and testing, not something mappers can reliably use on real maps. I'm not sure how my attempt to clarify that is "playing devil's advocate".

 

Based on what followed after my post, maybe the crashes are rare or already solved, I can't tell. But the very fact that I can't tell is still a problem.

 

So I guess my question is, is SEED in a reliable state that can be used by mappers in real missions, or is it still in a testing/rewriting phase where random problems and crashes are to be expected?

 

I'm trying to get a showcase map up, that's all.

[Tels]

I was considering using SEED in my next map, but when I see other mappers complaining about it crashing, and the developer who created it saying, "I am not sure why it crashes, and what the solution is," and "Have some plans for a rewrite... but this is in the near future."

 

I took that to mean that it needs to be rewritten so that maps don't crash. In other words, it sounds like a feature that is still in development and testing, not something mappers can reliably use on real maps. I'm not sure how my attempt to clarify that is "playing devil's advocate".

 

Based on what followed after my post, maybe the crashes are rare or already solved, I can't tell. But the very fact that I can't tell is still a problem.

 

So I guess my question is, is SEED in a reliable state that can be used by mappers in real missions, or is it still in a testing/rewriting phase where random problems and crashes are to be expected?

 

Okay, it seems I am really bad in the "presentation and communication" department

 

Let me explain here a bit:

 

First the crashes. There are a few things that can crash doom:

 

* when you build a model with more than a few hundred thousand tris (forgot the exact number).

* when you have too many things presented to the renderer. I am not exact if there is actually a limit (I think the ground zero developers said "10000 render entities" or so), and it is unclear if you ever reach it. I have successfully rendered hundred thousand (!) mushrooms (all in all a few million tries at the same time), and they were combined into a couple of thousand (if not ten thousands) entities do so.

* when you have too many things casting a shadow, or having too many shadow casting tris. The exact limit here is unknown (e.g. is the limit per map, per area, per visportaled room, or per view, or per model?)

* D3 has a tendency to crash if it runs out of memory. Instead of handling this gracefully, it usually simple crashes and burns.

* You can also exhaust the video memory this way, because things that get rendered multiple times are currently not rendered by "instancing" (e.g. the GPU just renders things 10x), but by cloning the models together (e.g. the GPU renders one giant model 1x). This, however, depends on the system. On a 512MByte GPU it might work, on a 256 GByte GPU it might crash. Depending on whatever else.

* Using more red than green textures on an nvidia card, on odd thuesdays when your CPU is watercooled. Okay, that one was made-up, but then, I have seen similiar bugs.

* there migth be other, hidden limits in the D3 engine

 

Now, you see, some of these limits can be worked around: for the "max x number of tris per model" thing SEED (actually: the ModelGenerator) prevents this by enforcing a limit on how many tris a model can have. This limit was found experimental (e.g. "build crazy models until it crashes, then scale them back until it no longer crashes"). Don't ask how many hours I spent making this work.

 

Likewise, SEED keeps a cap on the entities it creates, if you are running out of entities (the 8192 limit), it will cull unimportant entities far away and only spawn entities near you. Likewise, don't ask how many time was invested to make this little detail work transparently to the mapper...

 

 

Anyway, other limits are coded into the engine, cannot be prevented or known in advance - how could SEED know how many things are presented to the renderer, even if it only presents 9000 things, the rest of the level can present 1001 and make it top 10000, and crash).

 

However, all is not gloom and doom (no pun

 

Right now the only two limits that crash D3 that I know of are the memory limit, and the shadow casting (I have know about this, but usually it only happened when I did crazy things, like making a sea of cattails. So I regared it as "usually won't happen as nobody wants maps that big") My mistake. Anyway, I have not yet investigated why this is so or how it can prevented, tho. But eventually I will have time for this bug.

 

 

And in further defending my work against the accusation that "it is not ready to use because it can crash": You have to remember that SEED pushes the boundaries in the extreme. I mean, in regular cases D3 would not be able to handle 10000 trees (due to the entity limit), nor would it handle 10000 shadow casting entities, either (due to the entity limit alone, but also due to the "cannot have 10000 rendered single entities plus worldspawn). You can't even place 10000 trees in DR without it laggin' to hell and back with every mouse click...

 

There is also the issue that a lot of our vegetation is optimized to look good from close-up, but we didn't consider the "what happens if I have 5000 of them" usage case...

 

So I think the SEED system should be cut a little bit slack because it is doing things that are technically impossible, and it tries to achive that in real-time, with almost no delay, and tries to use not much memory in the process, either.

 

Most of the "failed" goals are due to the engine limit, not inherint in the SEED itself.

 

I know that last point is lost on potential users (afterall, they don't care why it fails, they just care it fails). And that's why I hope that open source D3 will let us overcome a lot of these limits and remove the crashes. Can't wait for this!

 

In the meantime, I'll do what I can.

 

 

Second point about the "rewrite". There are two things to consider:

 

 

* when D goes open source, we can hack the rendered and use instancing. So instead of building multiple models by gluing things together (which was never the plan, anyway), we can just tell the GPU "render this tree 1000 times with these parameters" and off it goes like a rocket. This *will* solve a few problems (like memory consumption, time to modify models) and thus also a few crashes as a side-effect. However, this cannot happen until D3 goes open.

 

* Also, the current SEED system is inefficient in the sense that each SEED is it's own entity, and controls only its own things. That means if you place 12 SEED entities producing grass of two variants, you get 12*3 entities (12 entities with grass #1, 12 entities with grass #2, and 12 SEED entities to control them, as the SEED needs to be around to cull/respawn entities). This happens even if the grass overlaps, and you could theoretically get away with 2 entities. The "rewrite" I was refering to was to restructure the system so that there is a SEED manager, and each SEED entity just registers its parameters to the manager. (You need the individual SEED entities so the mapper can place them, but you don't need them during run-time when the map runs). The manager than holds the data, and can cull/respawn entities. Not only would that eliminate multiple SEED entities, it would also allow cross-entity optimization. And with every entity that is not used, you save runtime (entities "think" and that needs a tiny percantage of CPU), and memory, and savegame size.

 

However, while this "rewrite" would solve a few features, and prevent a few crashes due to running out of resources, it is not something you need to "wait" for.

 

The map "Knighton Manor" shows you can use SEED right now. The definition of "use" tho greatly varies. Some people just want to add patches of vegetation or random game objects easily, and you definitely can do this now.

 

Other people want to create Crisis-Style outdoor jungles, and this is still somewhat problematic. Given the limitations of D3, however, I think it works quite okay. Your milage may vary as they say

[Tels]

Just forgot one thing: There is also the general problem that programmers make for poor testers and documenters. And that programmers aren't supposed to come up with usage cases, and make the models, too.

 

With SEED, it seems I had all these roles at once, and that is not good. Not only because it doesn't really work (I am blind for the limits of my work, and I am not a good documenter because I know how it works, but not what somebody needs to know when they don't have my knowledge), but also because it wastes too much bloody time. I spent ages making low-poly models (what a real modeller could do a on a weekend), and a lot of people still claim we don't need them, and there still seems nobody really making low-polies for our current models.

 

And yet, people wonder why D3 crashes if they place 5000 high-poly (2000+ tris) models in a map *sigh*

 

And that ties in with the "should we really use it yet?" question. If mappers aren't using it, then nobody will, then the issues either remain hidden (until someone tries it), or unsovled (because nobody really pushes for them to be resolved).

 

For me personally the most problematic thing is that the entire "oh I tested it and it crashes. So unuable." verdict comes X months after I lost energy and somewhat interest to work on it (you know, because nobody was using it), so I can't even dive in and fix things *right now* because I am currently occupied somewhere else.

 

Seems like a re-occuring theme to me and that makes me sad. Not sure if different communication could have fixed that. Maybe people where always waiting for it to be "finished"?

[Springheel]

And in further defending my work against the accusation that "it is not ready to use because it can crash": You have to remember that SEED pushes the boundaries in the extreme. I mean, in regular cases D3 would not be able to handle 10000 trees (due to the entity limit), nor would it handle 10000 shadow casting entities, either (due to the entity limit alone, but also due to the "cannot have 10000 rendered single entities plus worldspawn).

 

Ok, so the crashes aren't SEED-related, they're D3 related. Thanks for the explanation, that makes things much clearer.