The move came after terrorists attacked two mosques in the country's second largest city Christchurch and five guns were discovered, two of which are semi-automatic guns. Other weapons and firearms were also retrieved by the police.
It is understood that the killers of the terror attacks have legitimate gun license.
"Our gun law will be changed," the prime minister said.
So far three suspects are now in custody. They are all Australian citizens, but not in any watch list of New Zealand or Australia. A fourth arrest was not related to the killing and the person was then discharged.
A 28-year old man is due to appear on Christchurch District Court on Saturday morning.
The man has been charged with murder. Two others are still in custody.
It is reported that the man who has been charged with murder is named Brenton Tarrant. He is an Australian National who live streamed the killing via social media.
Christchurch Mayor Lianne Dalziel spoke to the public on Saturday morning, sending her condolences to the victims and their families and the people who are affected by the attacks.
"Our thoughts go to the victims and their families and everyone being affected," Dalziel said.
She also acknowledged the extraordinary response from the police and first responders.
"An attack on the Muslin community is an attack on us, on Christchurch and on New Zealand," Dalziel said.
The mayor believed that Christchurch would go through this together.
Major public events during the weekend have all been cancelled across New Zealand in the wake of the terror attacks.
At least 49 people were killed and 48 others injured in the terror attacks on two mosques in Christchurch on Friday.