With all the ballots counted, the Social Democrats have the most votes – just over 20% – but they do not have enough to form a government alone.
Analysts say the result could pave the way for another unstable coalition, with the second-placed ANO party in a powerful bargaining position.
The election has come after months of political turmoil.
The centre-right government of Petr Necas was brought down by a corruption scandal in June.
The country has been without a proper administration ever since – and is currently being governed by a caretaker cabinet of technocrats.
Tough talks ahead
Correspondents say that this election is likely to be followed by weeks of difficult negotiations.
The BBC’s Rob Cameron, in Prague, says the Social Democrats had hoped to win enough to run the country if they were supported or at least tolerated by the Communists.
But even together, they do not have enough votes to form a government, he says.
That opens the way for arduous talks on forming a coalition with some of the other parties in parliament.
Social Democrat leader Bohuslav Sobotka admitted the results of the election were “not what we expected,” but he told reporters he was ready to start negotiations with all parties.
Our correspondent says the real winner in this election is second-placed ANO, a new centrist party which campaigns against corruption and is run by a food and agriculture billionaire.
ANO has almost 19% of the vote, and the Communists have just over 15%.
The leader of ANO, Andrej Babis, has told reporters he could not envisage supporting a Social Democrat cabinet.
Czech vote results
Social Democrats: 20.45%
ANO 2011: 18.65%
Our correspondent says this election is being seen as a referendum on the relative popularity of the established parties – which have governed the Czech Republic for 20 years – and a crowd of colourful new parties.
Analysts say ANO was able to attract voters who had grown weary of the status quo, despite Mr Babis’ membership of the Communist Party before the 1989 revolution.
Meanwhile the Civic Democrats, the main party in the previous centre-right coalition, only received 7.7% of the vote.
“It’s a fatal loss,” the party’s acting chairman, Martin Kuba, told reporters.
The coalition broke down in June amid allegations of corruption, spying and abuse of power.
A close aide of Prime Minister Petr Necas, with whom he was having an affair, was arrested on suspicion of bribery and ordering a military intelligence agency to follow Mr Necas’ estranged wife.
Photo: The pressure is on for Bohuslav Sobotka to form a stable coalition