A decree read out on state television announced that Gen James Hoth Mai would be removed with immediate effect. No reason was given.
The country has been in turmoil since December. Last week rebels seized the oil hub of Bentiu.
Meanwhile the UN has accused the government of providing “erroneous information” regarding a massacre of hundreds of civilians in the town.
South Sudan Minister of Information Michael Lueth was wrong to tell reporters that residents seeking protection had been barred from entering a UN base, the UN mission said in a statement.
After rebel forces captured Bentiu on 15 and 16 April, they targeted hundreds of people who had taken refuge inside a mosque, a church and a hospital, and killed them because of their ethnicity, the UN Mission in South Sudan (Unmiss) said.
“At no point did the mission ever turn away any civilians who came to its camp to seek protection and instead opened its gates to all unarmed civilians,” it added.
Correspondents say last week’s killings are among the most shocking since the conflict began.
The rebels say the retreating government forces were responsible.
While no official reason was given for the removal of Gen James Hoth Mai on Wednesday, analysts say the military has suffered several setbacks in the north of the country in recent days.
Gen Paul Malong was named as the new general head of staff.
Analysts note that the sacked army chief hails from the same tribe as that of rebel leader and former Vice-President Riek Machar.
The conflict pits Mr Kiir, an ethnic Dinka, against Mr Machar, from the Nuer community.
A January ceasefire deal has failed to halt fighting that began in December after Mr Kiir accused Mr Machar of plotting to stage a coup.
More than a million people have since been forced from their homes since the violence started.
Talks between the two sides which were due to resume on Wednesday in neighbouring Ethiopia have been delayed until 27 April.
Bentiu, capital of the oil-rich Unity State, has changed hands several times during the conflict.
Control of the oilfields is crucial because South Sudan gets about 90% of its revenue from oil.
The UN expressed outrage at an attack last week on one of its camps in Bor in Jonglei State, saying it could “constitute a war crime”.
That attack by armed youths left at least 58 dead, including children.
Both Mr Kiir and Mr Machar have prominent supporters from various communities, but there have been numerous reports of rebels killing Dinkas and the army targeting Nuers.
The UN has about 8,500 peacekeepers in South Sudan, which became the world newest state after seceding from Sudan in 2011.