The Cabinet of Dr. Caligari (1920)

This is German Expressionism in one of it's most stylistic horror forms. I really like the music too. It adds to the general creepiness.
The Cabinet of Dr. Caligari was directed by Fritz Lang, who also did Metropolis. Fritz Lang did some seriously amazing stuff. Especially when film cameras were so limiting. There are only so many tricks you could do back then, but he really set the bar HIGH when it comes to inventing some crazy stuff. Metropolis is a perfect example of that, so if you ever watch it on DVD, invest some time viewing the extra features.
Anywho, back to Dr. Caligari. Pay attention to the sets and props. Are you as visually excited as I am?? This film really drives me nuts because it's so eerie. Everything about it makes you wonder how someone imagined it and what goes on in Lang's mind, how he wants to present all this to you, the viewer. Makes you feel special, right?
Although I DO understand, this may not be for everyone. Give it some time. It could grow on you.

Note:
If you're still interested in learning more, you can branch off into your own silent film interests. There's a lot out there so be prepared to spend some major time researching. A good way to learn some more about film is reading auto/biographies about different directors (Fritz Lang would be good to start out with) and actors. Also, keep open to suggestions. Maybe look around to see if there are any specialized theaters if you live in a large city; there are places that only show art films, places that only show foreign films, etc. Look in video rental stores under "Cult Classics". Look in the dollar bin at the crappy gas station. Be diligent, you'll eventually find what you're looking for.