So I was flipping through
The World of Ice & Fire and came across the Casterly Rock section. I always imagined it as a pretty regular castle, but it's nothing of the sort. More like a mountain. It's said to be three times the height of the Wall. Apparently not even dragonfire would do much damage on it. Pretty impressive stuff. I wonder how the hell Robb was planning to take it in the show.
Here's the art work from the book.