I have always said once there spots are gone they are good to go.
Also the meat is a whole lot better than any old mature trophy buck. I have never been a trophy buck hunter in my 50 years of hunting! Oh, I'll take one but I don't have the ego that I have to kill a giant buck, besides I can't eat the horns. All big horns I've ever taken end on as knife handles anyway.