Sour grapes.

There is nothing popular fiction hates more than somebody doing something everyone wants to do but can’t. Impossibility, when possible, becomes cast as immorality or immaturity or otherwise something arbitrarily undesirable.

To be a ghost is depressing and/or monstrous because when we die in real life we don’t stick around. Time travel overwrites reality with a worse version of the present because in real life we can’t change the past. Resurrection brings people back as monsters because in real life we can’t have our lost loved ones back. Immortality is sad and lonely and often requires you to do evil things to sustain it because in real life we can’t live forever. Traveling to alternate lifetimes where you’re more successful is emotionally hollow because you had the most important emotional stuff in your life all along and you wouldn’t trade that for the world.

These and other speculative crises always have to be fixed by making the fictional world abide be the limitations of the real one. Aren’t we so lucky that our world is randomly already like this?

Disclaimer that I don’t believe in ghosts, but in fiction at least, I think the usual implication is that a ghost is someone who hasn’t passed on correctly. A few people have brought up unfinished business already, but even in stories that don’t bring that up, ghosts are often people who died horribly, prematurely, and/or violently. Sometimes they’re explicitly under a curse keeping them from moving on.

Basically, the circumstances of their existence are wrong, and they’re stuck due to forces beyond their control. That’s kind of the tragedy of being a ghost; they’re often a whittled-down, corrupted version of their living self.

If you have ever had a dream where you can’t get something right. Like, in your dream you know you need to do something - you even know exactly what it is that you need to do. You have no excuse for not doing it - maybe you actually try to do it - and yet you keep finding it is as yet undone.

You eventually wake up, because you are alive and you are dreaming. But if you are dead, you are not alive, you cannot wake up. You have all of the agency that you had when you were dreaming, which is none. Until you woke up - you were helpless in your dream, try as you might. The dead are helpless in their dream as well, but they do not have the luxury of waking up.

So have some pity for the dead.

I believe it’s tied to the ides of the restless dead.

The spirit should move on after death, but some spirits get stuck between worlds. Often it’s related to the circumstances of their death. Maybe they had unfinished business, they died a particularly grousom death, or they were denied a proper burrial.

I’d imagine that the ghosts who ditched their unfinished business to explore the world beyond would be doing that exploring in a world beyond so we wouldn’t see them or anything.

I don’t believe in ghosts or psychic phenomena, but I do love the concept in fiction that ghosts aren’t actual human souls. Rather, they’re a sort of psychic “burn in.” If a living person experiences strong emotions, such as a prolonged period of grieving, or the incredible emotional intensities that come with being murder victim, those emotions can become embedded within a place. Do you grieve for a deceased partner, mourning for years, remembering key moments over and over? A reflection of that grief becomes embedded within reality in the location you experienced those emotions. When you die or leave, someone else can come into that place and experience a recording, a reflection, or echo of the emotions and memories you experienced.

Ghosts are effectively traumatic memories burned in to the fabric of the world. They don’t actually experience anything; they’re not conscious beings. They’re not souls looking to complete their business and move on. They’re simply psychic echoes. They’re imprints left on reality from very intense and painful emotions, particularly those experienced repeatedly over a long period of time.

This also explains why ghosts have a half life. Ever wonder why in the US, all the ghosts seem to be old timey white people from the 1800s or similar? Considering the total number of Native Americans that must have lived in what is now the US down the millennia, the vast, vast majority of ghosts should be Native American. But aside from the classic example of a disturbed native burial ground, Native American ghosts don’t show up much in fiction. It’s usually old timey white people.

The reason for this, in the imprint theory, is that like any imprint, ghosts tend to fade with time. Just as most footprints will slowly be eroded, the knots in the psychic fabric that ghosts represent slowly work themselves out over time. The ghosts people do experience tend to be from the last century or two, as most ghosts older than that have decayed below the level of human perception.

I think because in that framing being a ghost is a sad lonely thing. In the Christian tradition the ultimate reward is heaven and being in the presence of the lord. If one is stuck on earth it is similar to catholicism’s concept of purgatory.

Most of the ghosts at Hogwarts in Harry Potter seem to either actively enjoy being a ghost or at least not mind it and just carry on doing stuff (like assist the pupils)

That aside, plenty stories have the ghost relive the moments before their death involuntarily, over and over. So for them it’s nothing they can control.

But I’m with you: If I were not trapped in this deathloop (or bound to one location) and were able to be seen by other humans, I would definitely not be gloomy, once I got over the fact that I died and can no longer interact in all the usual ways with my spouse and dogs. Even if I could not do crazy telekinetic stuff, I could at least wander nature, haunt bad politicians and give my spouse inside knowledge, which she can possibly make a lot of money from to buy a house for us.

Popular beliefs influences people’s beliefs, which reinforces popular beliefs. Step back even farther from the question for a moment and ask, “why do you think of ghosts as dead human spirits at all?” That a “ghost” is some sort of dead human spirit is a concept that has been built into Western society for a long time. It is something we just accept in story telling and mythological belief systems because it’s been in them so long and is told to us via authoritative figures in our lives from an early age. To tell a story where a ghost is anything other than a dead human spirit or the echo of a dead human, makes people call bullshit on the story, because the story has broken a long standing societal expectation. Sure, some stories can get away with it, and more so in the modern age where we are starting to appreciate stories which subvert long standing expectations. But, we still tend to fall back on old tropes and devices which we can expect readers to understand, without having to spend too much time on building a world. It’s far easier to save the term “ghost” for something much like a dead human spirit and just create a new term when trying to describe something else.

Because we think of every life as a story, and every story needs narrative closure. So if someone’s life seems like an unfinished story we feel like there’s some kind of lingering agency trying to finish it.