In this talk we discuss the topological structure of the spectrum of almost periodic Schr\"odinger operators, both in one dimension and in higher dimensions. The problem is quite well understood in the one-dimensional case and the talk will briefly describe some of the known results. The question is significantly less well understood in higher dimensions. The Bethe-Sommerfeld conjecture for periodic potentials serves as a guiding principle for the different mechanisms and phenomena that should be expected to play a role. Passing from periodic to non-periodic almost periodic potentials, we discuss both positive and negative results in the spirit of the Bethe-Sommerfeld conjecture.