Dr. Madriz said you need to get to the doctor sooner than 45 if you’re having symptoms, which can include something as innocent as a hemorrhoid.