It would be the "first official large-scale" exchange of its kind since the beginning of the conflict, according to the U.N.
The prisoner swap deal was seen as a breakthrough during 2018 peace talks in Sweden. The Houthis and the internationally recognized government agreed then to several confidence-building measures, including a cease-fire in the strategic port city of Hodeida.
Implementation of the tentative peace plan stumbled amid ongoing military offensives and a deep-seated distrust between the two sides.
The conflict also has been a theater for the regional rivalry between Iran, which backs the Houthi rebels, and Saudi Arabia, which leads a military coalition supporting the internationally recognized Yemeni government.
The U.N. mission in Yemen said that both the rebels and Yemeni government had decided to "immediately begin with exchanging the lists for the upcoming release" of prisoners. Sunday's statement came after seven days of meetings between the two sides in Jordan's capital, Amman.
"Today the parties showed us that even with the growing challenges on the ground, the confidence they have been building can still yield positive results," the U.N. envoy Martin Griffiths said.
The talks were co-chaired by Griffiths' office and the International Committee of the Red Cross, ICRC.
Griffiths urged both parties to move forward with the agreed-upon prisoner exchange "with the utmost sense of urgency." He did not elaborate when they would start the exchange.
Franz Rauchenstein, the head of the ICRC in Yemen's capital, Sanaa, welcomed the step as "encouraging."
"Today, despite ongoing clashes, we saw that the parties have found common humanitarian ground that will allow many detainees to return to their loved ones," Rauchenstein said.
The war in Yemen has also spawned the world's worst humanitarian crisis, leaving millions suffering from food and medical shortages. It has killed over 100,000 people, including fighters and civilians, according to the Armed Conflict Location & Event Data Project, which tracks violence reports in Yemen.
Houthi spokesman Mohammed Abdel-Salam said they would release 1,400 prisoners including Saudis and Sudanese.
Yemeni Foreign Minister Mohammed Abdullah al-Hadrami said he welcomed the "phased agreement" to release prisoners, in a tweet Sunday.
The breakthrough in talks came after another weekend of violence in Yemen.
Strikes on Saturday killed more than 30 civilians in the mountainous northern Yemeni Jawf province, the U.N. humanitarian chief for the country said.
The Houthis accused the Saudi-led coalition of launching retaliatory airstrikes, after they shot down a coalition warplane over Jawf.
The renewed clashes threatened to overshadow the hopes raised by back-channel talks in the Gulf state of Oman between Saudi Arabia and the Houthis.
The talks focus on interim agreements, such as re-opening Yemen's main international airport in Sanaa, which was shut down by the Saudi-led coalition in 2016. In a sign of progress, two United Nations flights ferrying dozens of seriously ill Yemenis abroad for treatment took off last week from the rebel-held capital, the first since the start of the air blockade.
The conflict in the Arab world's poorest country erupted in 2014, when the Iran-allied Houthis seized the capital and much of the country's north. A Saudi-led coalition, determined to restore the authority of Yemeni President Abed Rabbo Hadi's government, launched a military intervention months later.