The thing is, often it's not very clear idea. Or the scope is very general. Like one of my goals is "I want to make pretty things". Or, one of the sidequest ideas I came up with, it was an impression, as I was listening to a song walking back home. As I started writing it down, it became more tangible, translated to gameplay, scripts, etc., but at first it was just an image of a boy, sitting on a riverbank, staring at the moonlight reflected in water
Creating for the audience is actually pretty important, more in games than in any other medium. All media are form of communication, whether you want to express yourself, and others will just be witnessess, or you want to communicate, make people learn or feel something. Games are different than other media here, because you can read entire book or watch a movie without understanding it, while you can't do that in games. Games will punish you for not understanding their rules, with not letting you progress. That's why level design is mostly about communication: teaching and rewarding. And, as in every craft and medium, there are layers. There are certain things you make that will be noticeable to the audience. There will be technical things that your peers will notice and praise you for it (or dismiss you as a craftsman, if they find it cheap or low-quality). And the last layer is these few tiny things you really wanted to make, and you did, but probably noone but you will ever notice them Now we may look down upon pop music or AAA games, but their production quality is top-notch, as is the focus on the audience. It's just the content that is pretty shallow.
As for the setting, veteran level designers could probably translate any greybox level to anything, but even they will choose something that will facilitate what's established for the story and gameplay, so the two can meet in the environment. And the environment will detemine the gameplay to some degree, like dimensions, distance between ledges, walls, covers, lines of sight, etc. Recreating mansion gameplay in the forest could be an interesting excercise, but it is going against the grain a bit, isn't it
Last but not least, IMO this thread is more like "tips and tricks", rather than some general way-of-life Yoda-style statements like "do or do not, there is no try", and that's why the OP question is worth adressing, as everyone has their own approach, tricks, and ideas.