Motion Controllers (Vive or Oculus) without plug-ins (SteamVR or Oculus Tools)

Is it possible to make use of the motion controllers (Vive) or the Oculus Touch controllers with native Unity 5.5 - i.e. without the SteamVR asset nor the Oculus Tools?

I noticed that the Vive HMD works fine without SteamVR, but I'm not sure how and where to add controller support, as they don't "simply work".

Any pointers would be greatly appreciated.