Just to let you know that it IS possible: I've been married for 12 years now to a wonderful woman who has never had a problem with any of my hobbies, including video games. On occasion we play them together(Mario kart, puzzlers, etc.), but for the most part, it's my thing. The key is that I rarely, if ever, play them during the day. It's almost always at night after the wife and kid have gone to bed. I'm actually way more nuts about the whole gardening thing, but who in their right mind would complain about that?
No matter how much time I spend on gardening, gaming, watching movies, etc. I always make sure to spend equal time hanging with my son and/or my wife. No one should ever have to be reminded to do such things.