For about 800 years there has been a building on the islet by the “drag”. From the original palace over the medieval castle to the current baroque style, Dragsholm Castle has had an influence on and been influenced by changing times and the surrounding community. Today, Dragsholm Castle has restaurant and hotel facilities.

Dragsholm Slot is one of the oldest secular buildings in Denmark. The original Dragsholm Castle was built around 1215 by the Bishop of Roskilde. During the Middle Ages, the building was modified from the original palace to a fortified castle. During the Count's Feud (1534–36) (Grevens Fejde) it was so strong that it was the only castle on Zealand to withstand the armies of Count Christoffer.