Severance pay

Why severance pay is considered part of wages por income tax purpose?. This is a benefit that a company give to employees when they reduce or lay off personnel. I was laid off in June 2004 at the age of 61 and 30 years of work. The company provided me a severance pay for one year starting July 2004 until July 2005 with no labor involvement. On October 2004 I got interviewed by a Social Security employee representative in regards my benefits in 2005 when I became 62 Years old. I presented all the documentation he asked for included the severance papers that shows how much I was receiving in benefits without labor involvement. He read and took copies of all documentation I presented. In my interview I mentioned him if the severance pay would be an obstacle to receive my social security benefits. He said was no problem and no conflict of interest. He scheduled to received my Social security benefit starting March 2005. On August 31, 2008 I received a letter from the Social Security Administration in which they stated that I was working that year which is not true and was paying taxes and Social security taxes in 2005. Now they demand that I have to reimburse around 13,000 Dollars. I stopped working in June 2004 ,since then I never worked. Why an employee or representative of the Social Security that I got interviewed in October 2004 didn't advice in the correct manner since I was ignorant of this matter?