He had held final discussions with both sides in Geneva in a last-ditch bid to break the stalemate between the Syrian government and opposition.
Mr Brahimi said the main sticking point was the government’s refusal to talk about a transitional governing body.
No date has been set for a third round of talks.
The conflict in Syria has claimed more than 100,000 lives since March 2011.
Some 9.5 million people have been forced to flee their homes.
The discussions on Saturday morning lasted just 27 minutes, after which Mr Brahimi emerged looking tired to address reporters, the BBC’s Imogen Foulkes reports from Geneva.
Mr Brahimi apologised to the Syrian people, admitting that the talks “had not come out with very much”.
Although the two sides had agreed on an agenda for the next round, Mr Brahimi said the government had rejected his suggestion to begin a third round by spending a day focusing on combating violence and terrorism, and a second day discussing a transitional government.
Mr Brahimi said the government’s stance “raises the suspicion of the opposition that the government doesn’t want to discuss [a transitional government] at all”.
But the government’s chief negotiator, Bashar al-Jaafari, stressed that “terrorism” – rebel violence – must be dealt with fully first.
“Once you have an agenda you should respect the agenda fully without any interpretation or misinterpretation… we said that we cannot move from… item one to item two or item three or item four without fully considering this item and concluding by a common vision of this item by the two sides, something that the other side objected to.”
Opposition spokesman Louay Safi again insisted on discussion of a transitional government that does not include President Bashar al-Assad – something the government rejects.
“A third round without talking about transition would be a waste of time,” Mr Safi said.
Mr Brahimi said both sides needed “to go back to their base” for consultations and to think about whether they wanted the process to continue or not.
So far, the only agreement in the latest negotiations was to allow civilians to leave the besieged city of Homs and aid to enter.
Correspondents say at least 5,000 people are believed to have died since the first round of the Geneva talks began on 22 January.
US President Barack Obama said on Friday that he was considering ways of putting more pressure on President Assad, though he did not expect any resolution in the short term.
Speaking in California, where he was meeting Jordan’s King Abdullah, he said: “There will be some intermediate steps that we can take applying more pressure to the Assad regime and we are going to be continuing to work with all the parties concerned to try to move forward on a diplomatic solution.”
Mr Obama did not disclose what steps he has under consideration.
Error: No connected account.
Please go to the Instagram Feed settings page to connect an account.