Well the thing is, the RX 480 is 'optimized' as an entry level VR card. It hits the minimum requirements put out by Oculus and HTC as a means to open up the VR market. I have a R9 390 that I could use that will also meet the minimum VR requirements. The 1080 is also optimized for VR with tech to allow for both frames needed in a VR headset to be generated in a single pass through. Currently if you want the best VR experience the 1080 is it.
It's sort of like driving cars. The 480 will be like the Toyota Camry, reliable as heck, and will get you where you need to be on a budget. The 1080 is more like the BWM M3, still get to the same place, but it looks a heck of a lot better while doing it.
I had the money at the time, and I want that performance for the best experience.