Did I make a mistake not signing up at 65 ??

I am 70 yrs old and retired collecting pension from my employer, I have Blue Cross Blue Shield insurance and due to the increasing costs of my medications I checked into signing up for Medicare and will be looking for a supplemental health plan.........when I called the social security office about signing up for medicare I was told since I didn't sign up at the age of 65, my medicare premiums monthly would be double because I would be penalized for the last 5 years. Is this true ? When I consulted with my employer's benefits office at my retirement I was led to believe since I had my own insurance I would not need to sign up at the age of 65........Please advise ???