Well, there are a couple of sources:
1. Water getting into the assembly and forming a "dew" on the inside (although that's pretty apparent)
2. Road debris hitting and scratching the lens. That's why you can sand it down to a relatively smooth surface again, apply some Meguires, and they look like new.