Yes if you just want to lift for an hour — it is the plain open-access gym in Sag Harbor village, and the $40 day pass means you never have to join anything.
gym · Sag Harbor
Sag Harbor Gym
Hamptons Gym Corp's waterfront branch — 14,000 sq ft of cardio and free weights under a glass atrium, with a dry sauna. The day pass is valid in Southampton too.