Castlebay takes its name from the island fortress of Kisimul Castle, first built by the MacNeil clan in the 11th century. A short boat trip (weather permitting) takes you out to the island, where you can explore the fortifications and soak up the view from the battlements.
The castle was restored in the 20th century by American architect Robert MacNeil, who became the 45th clan chief; he gifted the castle to Historic Scotland in 2000 for an annual rent of £1 and a bottle of whisky (Talisker single malt, if you’re interested).