Zawawi Abdullah, a senior official with the Malaysian Maritime Enforcement Agency, said initial police information showed that only one of 25 Rohingya men who jumped off the boat had managed to swim ashore.
The lone survivor sought help from police late Saturday and has been detained for further investigation, Zawawi said.
Hundreds of thousands of Muslim Rohingya have fled Myanmar due to a military crackdown, and many live in densely crowded refugee camps in Bangladesh.
Muslim-dominated Malaysia has been a common destination of boats arranged by traffickers who promise the refugees a better life abroad.
Zawawi said two vessels and an aircraft were deployed in the search. He said the agency will also alert local fishermen and authorities in neighboring Thailand to assist in the search. He didn't provide further details on the boat.
Last month, Malaysian authorities detained 269 Rohingya refugees and found a body on board a suspected smuggling boat off Langkawi that they said had been intentionally damaged so it couldn’t be turned back to sea.
The Malaysian Maritime Enforcement Agency later said that the 269 were transferred from a larger vessel on which dozens of people were believed to have died and their bodies thrown into the sea.
Prime Minister Muhyiddin Yassin said Malaysia cannot accept any more Rohingya refugees as it struggles with an economic slowdown due to the coronavirus pandemic. But he said those already in the country will not be sent back out to sea.
Malaysia is home to over 2 million illegal immigrants as well as some 180,000 refugees and asylum seekers — including 101,000 Rohingya — who are registered with the U.N. refugee agency.