I never read the manga. Maybe I should give it a shot, but I do know for a fact that the anime was loaded with filler. They actually ended the anime series several years before the manga finished because it just wasn't doing well and wasn't interesting at all. Compounding that was an experience I had at a book store back when the RuroKen manga was first being published in English. A crazed Inuyasha fan was also browsing the manga after having talked to me about Inuyasha for five minutes. When she passed the first volume of RuroKen she said that Kenshin "looked just like Inuyasha," and it made my eighteen year old self mad for some reason I don't remember. I know it's not fair, but I've had a distaste for Inuyasha ever since.
Don't bother. No, reallly, and I'm saying this als one of those teenaged girls that loved the series back in the day.*
It's one of those series that's the most fun when you get on the ride at the right time/age. Outside of that context it's just one of many fun shonenseries that went on for way too long.
Oh, and yes, the fans were (and probably still are) still annoying. I can only hope I wasn't too bad, but then again, my awfulness was probably contained by not having anyone to talk to.
Have a funny Youtube video
*To the point where I bought the first volumes in three different languages. Started out in German, and when they announced it would be available in Dutch, I started collecting that edition. Then the Dutch manga industry died/was murdered by the Japanese publishers, and I turned my back on the series for a while. However, Viz suddenly started publishing those omnibusses and I was feeling nostalgic and, yeah, I finally own the whole series now.