The 36-year-old fast bowler, who was forced out of the World Cup this summer by a shoulder injury, made his Test debut in 2004 and took 439 wickets.
He will continue to be available for one-day and T20 internationals.
“It’s terrible to consider never playing another Test again but what’s more terrifying is the thought of never playing again at all,” he said.
Steyn is eighth in the all-time Test bowling list and took 26 five-wicket hauls, with a best of 7-51.
“Today I walk away from a format of the game I love so much,” he added.
“In my opinion Test cricket is the best version of this game. It tests you mentally, physically, emotionally.
“I will be focusing on ODIs and T20s for the rest of my career to maximise my full potential and ensure my longevity in this sport.”