Everyone has pretty much hit the nail head in that it's your immediate back drop. I don't do tree stands any longer either and all my set ups are 10-16' tripods tucked back into trees that I trim out just enough to get the platform back into the foliage and as close to the main trunk of the tree as possible. My favorites are tucked into live oak thickets.
Being still and deliberate with your movement is key. You will be sitting there and suddenly a deer is right underneath you. Be patient and pick your time to get your bow off the hanger and onto your knee. Be patient and pick the right time to draw also. At that low of height any deer you don't see that is behind you or out of your peripheral vision is going to bust your movement. It will happen but 90% of the time they will come back because they don't really know what they saw. Just stay still and don't do the "Oh chit I just busted all the deer outta here" tree dance.
Of course a lot of the times they come back they work back down wind of you and then you are really screwed.
BTW, I wear scent-loc leafy wear pants (don't like the top) and it seems to help further break your outline. I don't get busted very often.