Torre turned down a $5 million, one-year contract Thursday that still would have made him the highest paid manager in baseball.
Bench coach Don Mattingly is the leading contender to replace Torre, who led the Yankees to 12 straight playoff appearances and won four World Series championships. Yankees broadcaster Joe Girardi, the NL Manager of the Year with Florida in 2006, is another top contender.
"It's a difficult day," general manager Brian Cashman said.
But it was one Yankees' fans could see coming.
After losing the first two playoff games to Cleveland, owner George Steinbrenner said he didn't think Torre would be back if the Yankees didn't advance. They lost in four games.
Torre, who took over the team to start the 1996 season, made his decision after traveling fom New York to the team's spring training complex in Tampa, Fla. He was accompanied by Cashman and chief operating officer Lonn Tros.
"It is now time for the New York Yankees to move forward," team president Randy Levine said.
Torre made $7.5 million this year, the final season of a $19.2 million, three-year contract. His new deal would have included substantial bonuses for each round of the playoffs the team reached.
"We felt we needed to go to a performance-based mode," Levine said. "We thought it was very fair. It clearly was at the top of the market, but we respect Joe's decision."