Thank you so much for all the responses. I think I will go ahead and bring a down jacket, just in case it does get chilly. I am also a person who doesn't like to be cold.

I will be making a 5 day, 4 night trip next week to the island. I am wondering if I should bring a down jacket/sweater with me. I was looking at prices for some and seems like most are $150+. Could I get away with bringing long sleeve shirts and a fleece or should I look into buying a down jacket ...