I'm not sure I understand the controller issue. If the handheld is connected to the TV, isn't it the controller itself? When I connected my PSP to my TV I didn't use a separate controller; my PSP controlled the action.
Still, options are good. Play on the go, or play at home. I find myself playing games on my iPad and 3DS more than my PS3.
As for brand loyalty, I already answered that. my brand loyalty pertains more to clothing than anything else. Some brands of clothing are simply better sized and shaped to my body type.
The smart thing to do would be to have the handheld as de default controller, but offer a wireless controller as an alternative/to be used for multiplayer. Sold separately, of course.
Basically, the thing would be like a Vita TV with a screen and buttons, I guess.