Oblivion probably has my favourite Thieves Guild and Fighters Guild in the series. Dark Brotherhood was great too. Probably had more fun with Oblivion's Dark Brotherhood quest line than I did with any other faction in the series.
My main issues with the game were the dungeons and enemy level scaling. Wasn't a fan of how Bandits eventually all had glass armour and certain enemies disappeared from the game entirely. I like Morrowind's method the best although I do like how Skyrim took the Oblivion route and improved it.
I replayed the game earlier this year and really liked something I didn't really appreciate on my first playthrough. The game is so damn colourful. In a world filled with brown and grey games, it really stood out. The game still looks great if you ignore character models.