(AFP) - A high-rise building under construction collapsed in Nigeria's commercial capital Lagos on Monday, killing at least four people with dozens more feared trapped inside the rubble.

A yellow excavator pushed away concrete slabs to search though the wreckage of the 21-floor building in Lagos's wealthy Ikoyi residential and business district, AFP correspondents at the scene said.

Rescue officials said many workers were caught inside the building when it crumbled, though they could not confirm the number of people trapped inside.

Ibrahim Farinloye of the National Emergency Management Agency (NEMA) said four people had been rescued so far and four bodies recovered from the site.

Wisdom John, 28, a bricklayer, told AFP he escaped with just a few cuts because he had been on the ground floor when the building collapsed into a pile of concrete, its floors sandwiched together.

"There was more than 50 working today and the manager too," he said, sitting in an ambulance getting treated. "We just ran out."

The Ikoyi area is one of the wealthier residential and business districts in Lagos, Nigeria's densely populated major commercial city.

Near the collapse site, soldiers kept back a crowd of onlookers watching the rescue operation.

Dozens of angry local residents had gathered at the site soon after the collapse, many crying and voicing frustration over the slow pace of the rescue efforts.

Lagos State police commissioner Hakeem Olusegun Odumosu it was still too early to determine the cause of the collapse.

"Many workers are trapped under the rubble," said Femi Oke-Osanyintolu, general manager of Lagos State emergency management agency.

Four other construction workers at the site told AFP dozens of their colleagues were inside when the building crumbled.

"Like 40 people were inside, I see 10 bodies because I climbed up," said Peter Ajagbe, 26, one local worker on the site.

"One of my partners is dead."

Taiwo Sule, 21, another worker, said he had seen five bodies on top of the collapsed building, where he had tried to help recover them.

An AFP reporter at the scene saw at least one person being pulled unconscious from the rubble.

Building collapses are common in Lagos and other parts of Africa's most populous country because of the use of sub-standard materials, negligence and the flouting of construction regulations.

In one of the worst building disasters, more than 100 people, mostly South Africans, died when a church guesthouse crumbled in Lagos in 2014.

© Agence France-Presse