Multiple reasons, I think. The first one is that it's safe. It's something players have come to expect from their favorite genre, if they don't like it they'll atleast tolerate it. Developing games for modern systems is expensive, so most developers want to play it safe. It makes sense to keep something players recognize in every new game. Developers that want to innovate usually try something new with either the story/setting/characters or gameplay, not both at once.
Another reason is that some writers may be stuck in there comfortzone. They write one type of story and write it well (well, sorta, kinda, maybe...?), and as long as developers (who may be the same people as the writers!) are willing to use those stories, writers are allowed to remain safely in their zone.
A select few may be telling the same story again and again for personal reasons. They have a certain story in their head they can't turn into a game the way they'd like to due to budget restraints, time restraints and so on. The final product is never quite what they want it to be, so they keep trying. I think that's how we went from Xenogears to Xenoblade.