Did I go too far ?

I am working on an easy project, which is simply a scene downloaded from the asset store.
While the movements on the screen seem ok, on the Vive HMD it's awfully shaky.
My GPU is an R9 390 with 8GB VRAM, is the amount of polygons in the attached image the issue ?
Thanks.