What your grip on your bow is made out of should not be an issue at all because if you actually grip the bow you will most likely torque it making shots inconsistent. Hand should be open when shooting or shoot like I do when making a shot and touch your thumb with your middle finger making it impossible to torque your bow. Well maybe not impossible, you may have small hands or short fingers.
Wait a minute, I resemble that remark.
While the comments about the bow grip not being what causes torque may be accurate it is also not going to help you if everytime you put the fat of your hand up against it you aren't comfortable. I know the metal grip against my hand isn't. I know the rubber grip always felt like it was sticking and not letting my hand settle in well. I know I'm no Levi Morgan but I've shot a few bows for a few years and have figured out that what feels comfortable I'm more likely to have confidence in and shoot better. Not to mention shoot more often and enjoy the experience more.