Board Game Ouroboros

The first board game I recall (well, that I recall reading about, anyway) with a cooperative, almost roleplaying sort of premise was Shadows Over Camelot, released in 2005.  In it, you play the knights of the Round Table, questing for Camelot.  But!  One of you may be a traitor who seeks to betray Camelot.  Note that "maybe."  The traitor is randomized such that it is a different character each time, and it is entirely possible that no one is the traitor.  This last part is the most important mechanic, as the uncertainty makes everything more interesting.  In addition to this, there is a vague storytelling aspect as quests are completed for the glory of Camelot.

I believe Battlestar Galactica (2008) has a similar uncertainty with its traitor, but halfway through the game, you check for Cylons again, making it much more likely that you'll encounter a traitor at some point.  Also, outing the traitor doesn't end the problem — then you're just openly opposing the traitor.  Betrayal at House on the Hill (2004) similarly forecasts your inevitable doom — the traitor is inevitable, but isn't determined until the game's climax.  Basically, you're wandering around a randomly-generated mansion with a deep-cover agent who will eventually fly into a cackling rage and enact some grand scheme.  Until that time, though, everyone is working together.  As such, it lacks the same level of paranoia, but that's replaced with a sense of you bastard! instead.

A more recent offering (2010) is Castle Ravenloft, basically a cooperative dungeon crawl with a D&D fourth edition-lite flair.  It lacks the traitor mechanic, but is heavily randomized (basically like an analog version of dungeon-building in Diablo), so there's a lot of replayability.

To the point: last night, I played Mansions of Madness (2011), not to be confused with the classic Call of Cthulhu module of the same name.  It doesn't feature a traitor or a betrayal (though it's possible that other scenarios offer such), but it does form a more narrative experience than, say, Candy Land (though I suppose that's open to debate).  It also offers a very familiar mechanic: one of the players takes the role of the Keeper, and runs all the monsters (as well as the scripted events and random events of the night), while everyone else picks a character and navigates the mansion of the scenario.  This caused a bit of confusion at first, as everyone at the table was a veteran roleplayer, and we immediately fell into our appropriate roles.

The board game did not reward this behavior.

In a horror roleplaying game, it is best to avoid triggering the horror tropes, primarily by sticking together and staying alert.  Mansions of Madness, however, is on the clock; you need to split the party to uncover clues as quickly as possible so that you can get a headstart on stopping whatever horrible thing is going to happen.  We didn't, and suffered an appropriately Lovecraftian pyrrhic victory — the summoned shoggoth evaporated, but all but one of the party lay dead.

Mostly, though, this is an interesting loop.  Board games like chess form primitive wargames; the initial roleplaying offerings of the 1970s came out of wargames (most historians of the hobby trace its lineage directly back to the Little Wars of H. G. Wells).  In turn, the Call of Cthulhu RPG helped beget the board game Arkham Horror (2005, though an original version was published in 1987), which prompted Mansions of Madness — which emulates the structure of the original Call of Cthulhu RPG.

More interesting than that is probably an examination of Mansions of Madness as a roleplaying game.  If I describe what happened last night, I probably wouldn't just leave it at, "Oh, we played a board game, it'll probably be smoother when we know the rules better."  I'd probably tell you about how the private investigator wasted zombies with his twin automatics (except for the zombie that set itself on fire and died), or how I threw a fire extinguisher at the charging fanatic and completely missed him, or how the weird flapper psychic kept binding all the monsters in place, or how the nun was the only survivor.  It has more pieces than a roleplaying game, but the overall mechanics are simpler — exploration is a fairly simple affair (you either find something, find nothing, have to fight something, or have to solve some form of puzzle), and combat moves pretty quickly.

In fact, that can be said for most of these games — these board games have much of the complexity of a roleplaying game while telling a story in a much more focused arc.  They also have interesting ideas of adventure design, and even manage to randomize the map with simple, cardboard tiles.  This is, perhaps, something to consider when running your next game, both from a mechanical perspective and a storyteller one.