Why fps drop to half when i look the sky?

I just tested htc vive, with a very small terrain and using skybox or color in camera when i look up to the sky fps drop from around 100 to around 45.