I generally have a rule when it comes to my games...
If it's an RPG I will play on the "normal" setting to experience the game as the developers intended. They will tend to last roughly 30-50 hours anyway.
If it isn't an RPG (recent examples being Vanquish, Deadly Premonition, God of War III) I will bump up the difficulty. Those types of games last me longer if it's harder. I can't stand looking forward to game for so long only to have it beaten in 8-10 hours! I enjoy them much more that way.
I do the same with rpgs, but the opposite with non-rpgs. Those go to the easy setting pretty much without exception. Otherwise, I am unable to get very far into the games. For me difficulty does not equal fun.
To contrast that, I recently played through Metroid: Other M, which did not have a difficulty setting. I found the game challenging(especially some of the later bosses), but totally doable.
I'm playing Lords of Shadow now and am finding the "Squire" setting to be plenty challenging for me. I can't imagine attempting any of the higher settings.