Are you launching Orbiter.exe or Orbiter_NG.exe. you'll want the NG one. it sounds like a DX7 max resolution issue. With orbiterNG (no graphics), you can enable the D3D9 client.
There was an old (20 years now?) guide, called "Go play in space". It's still on the Orbiter wiki and a few other...