Andriy Parubiy said they wanted to move them “quickly and efficiently” to mainland Ukraine.
Earlier, pro-Russian forces seized two naval bases – including Ukraine navy’s headquarters – in Crimea.
It comes a day after Crimean leaders signed a treaty with Moscow absorbing the peninsula into Russia.
A referendum in Crimea on Sunday, approving its split from Ukraine, came nearly a month after Kiev’s pro-Russian President Viktor Yanukovych was replaced by Western-leaning interim authorities.
Mr Parubiy, in a news conference, set out more details on Kiev’s position in light of the events in Crimea.
“We are developing a plan that would enable us not only to withdraw servicemen, but also members of their families in Crimea, so that they could be quickly and efficiently moved to mainland Ukraine,” he said.
He also said the arrangements were now being set up to introduce visas for Russian nationals travelling to Ukraine.