The landslide, which came after heavy monsoon rains, engulfed about 140 houses in Badulla district.
Ten bodies have so far been found but the rescue efforts have been suspended for the night.
Bad weather is also hampering the rescue operations. Heavy rain has hit much of the country over the past few weeks.
Mudslide warnings had been issued for many areas,
The deadly mudslide hit the Meeriyabedda tea plantation near the town of Haldummulla, about 200 km (120 miles) east of the capital Colombo, on Wednesday at 07:30 local time (02:30 GMT).
“More than 100 people are believed to have died,” Disaster Management Minister Mahinda Amaraweera told Reuters news agency.
“We have suspended the rescue operations because of darkness and inclement weather. There is also a threat of further landslides.”
‘Thirty feet of mud’
About 500 military personnel are said to be involved in the rescue operation.
Sri Lankan President Mahinda Rajapaksa tweeted that heavy machinery had been deployed to speed up the efforts.
Local MP Udith Lokubandara told the BBC that many parents had returned home after leaving their children at school when the landslip happened.
“It is a very sorry situation because there are many children who have become orphans,” he said.
Maj Gen Mano Perera, the top military official in the area, told the AFP news agency that “some houses have been buried in 30 ft (9m) of mud”.
A driver who works on the plantation, P Arumugam, told Reuters: “Everything that I saw yesterday I could not see today – buildings, the temple and shops had all disappeared. I could only see mud everywhere.”
One woman who survived told local media: “I was under the rubble and some people took me out… my mother and aunt have died.”
Officials say an area of about two kilometres (one mile) has been affected by the landslide, Sri Lankan newspaper Daily Mirror reports.
Sections of several national highways have been washed away by the rains, reports say.
In June, monsoon rains triggered landslides in Sri Lanka that killed at least 22 people and forced thousands from their homes.
Monsoon rains are caused by winds in the Indian Ocean and south Asia. They bring about wet and dry seasons in much of the region, and have a large impact on local ecosystems.