The retrieval comes a day after the first piece of the so-called black box, the flight data recorder, was also found and brought to shore.
The aircraft with 162 people on board disappeared between Surabaya in Indonesia and Singapore on 28 December.
The two devices will help investigators understand more about what went wrong.
Forty-eight bodies have been recovered so far, but most of the victims are believed to still be inside the fuselage, which has not been found.
Search teams had said on Monday that they had spotted what they believed was the voice recorder, trapped under layers of metal and debris on the sea floor.
Search and rescue agency chiefs would not immediately confirm the voice recorder find – a formal announcement is expected later.
But an official involved in the search told reporters the device was now on board Indonesia’s Banda Aceh warship, in the Java Sea.
The BBC’s Karishma Vaswani said the device – which records all conversations between the pilots – was being taken to Jakarta, where it will be analysed by aviation experts.
The flight data recorder – holding information about the speed at which the plane was travelling, its altitude and other technical information – is already in the capital.
Flight recorders are designed to survive a crash and being submerged in water. They contain underwater locator beacons which emit so-called “pings” for at least 30 days.
These pings were detected by search vessels at the weekend but divers were prevented from going down to find them by strong currents and high waves.
The AirAsia was 42 minutes into its short flight to Singapore when it vanished from radar.
The cause of the crash is not yet known, but bad weather is thought to have been a factor. The pilot’s last communication was a request for permission to change course to avoid a storm.
But it has also emerged that AirAsia may not have had clearance to fly the Surabaya-Singapore route on the day in question. Its licence for the route has now been suspended.