Construction of the medieval castle on a travertine hill dates back to the beginning of the 12th century. The oldest written reference to the castle is from 1120. At the beginning it was a boundary fort placed at the northern frontier of an early feudal Old Hungarian state. Afterwards, it became the seat of the head of the Spiš region for many centuries.
In the second half of the 15th century the reconstruction of the castle fell upon its new owner Štefan Zápoľský whose intention was to remake it into a stately aristocratic residence. He had made a palace, a knight hall and chapel of St Elisabeth in the castle. His son Ján, later King of Hungary was born at the Castle.