Friends just got back from Palm Beach. I've only been there once, for a lunch of stone crabs (which are overrated).
It's a narrow barrier reef, just a few feet above sea level but nobody seems worried about that at all. There is a beautiful, tony estate area of the island with magnificent tropical and subtropical gardens like Casa Phippsberger:
and other areas with less exclusive, more normal life, condos, schools, etc, and not all very expensive.
Of course, Trump's Mar-a-Lago is there. His club was the first in Pal Beach to welcome Jews and blacks.
Here's See over the Hedges into the Gorgeous Private Gardens of Palm Beach
From what I gather, few people go to the beach there. There are heated pools. In winter, the sea is too cool for most people to swim (about 55 degrees F).