It's the difference between your girlfriend, fiance, wife watching you do your favorite activitiy and actually DOING it.
I think deep down most women that have a real commitment with their man really want to take part in whatever their significant other is doing. Most of my ex's could not play video games for the life of them, but the ones that cared would watch me play video games because they knew I enjoyed them so much.
Now if I was playing a game and I was playing it with a girl ONLINE and she knew lots of gaming trivia and odd factoids oh yeah it' s gonna make the onlooker very jealous. They feel you have much more of a bond with them.
So basically you riding is one step above them just being somewhat of an "onlooker". You can talk the talk and chat the chat and a lot of women become insecure about that, especially if you are attractive as you say you are that just makes it worse.
Personally for me, I like powerful older women, that's why I'm very happy with my fiance. She doesn't ride motorcycles but she has a commanding and powerful presence and is very beautiful and successful. But when she's with me she has this soft lilting voice and we do very goofy things.
It's like having a Tiger and a Kitten at the same time

You're like the Ghost in the Shell/Tom Boy that I think most men want.
I just hope you don't let anyone else's nasty stares or envious glares phase you. Theres a shortage of women out there like you!
(Edit: Spelling Ugh I just woke up)