Astigmatism is one of the types of focusing problems, the others of which are long and shortsightedness. In astigmatism, light is focused differently depending on which direction the light rays are shining as they pass through a lens.
Like other focusing problems, astigmatism affects vision in a way proportional to its degree. The higher the astigmatism, the blurrer the resulting vision.
One group of people with high astigmatism would need to wear glasses all the time-that would be children below the age of about 9 or 10 years old with astigmatism of greater than about 1.50D. If these children don't wear their glasses, the constantly blurred vision could cause lazy eye (amblyopia).
For older people, the glasses are purely a visual aid-ie they help to provide clear vision, but do not otherwise affect the health of the eye whether they are worn or not. So it is up to them whether they wear the glasses all the time or not-without glasses, vision is blurry, with glasses vision is clear-and that's about it.
Regular, stable astigmatism is not really a disease-it is just who you are and reflects the shape of your eyeball. But if you do not want to wear glasses or contact lenses, then the only other way to get rid of it would be through refractive surgery ie LASIK, PRK or ICL.