I've noticed this too. I think its a combination of a couple of things. One is that when you're driving and floor it you know exactly when youre gonna start accelerating so you brace yourself for it a little. Another is you're not in control so it seems scarier. And finally, you're just sitting back and relaxing in the passenger seat whereas when youre driving you have the wheel to hold onto for support and are concentrating on driving itself.