"Yesterday afternoon, Fulgence Kayishema - one of the world's most wanted genocide fugitives - was arrested in Paarl, South Africa in a joint operation," the International Residual Mechanism for Criminal Tribunals (MICT) said in a statement.
Kayishema, a former judicial police inspector, faces charges of genocide, complicity in and conspiracy to commit genocide, and crimes against humanity.
He has been on the run since July 2001, according to the MICT website.
He and others allegedly murdered more than 2,000 Tutsi men, women and children who had taken refuge in a Catholic church in Nyange in Kivumu district.
"Kayishema directly participated in the planning and execution of this massacre, including by procuring and distributing petrol to burn down the church with the refugees inside," the statement said.
"When this failed, Kayishema and others used a bulldozer to collapse the church, burying and killing the refugees inside. Kayishema and others then supervised the transfer of corpses from the church grounds into mass graves over the next approximately two days," the statement added.
MICT, which is based in The Hague and the Tanzanian city of Arusha, in 2015 took over the work of the International Criminal Tribunal for Rwanda (ICTR), which the United Nations set up following the genocide.
His arrest was carried out in a joint operation by MICT's Fugitive Tracking Team and the South African authorities, the statement said.
"Fulgence Kayishema was a fugitive for more than twenty years. His arrest ensures that he will finally face justice for his alleged crimes," MICT Chief Prosecutor Serge Brammertz said in the statement.
Around 800,000 Rwandans, most of them ethnic Tutsis, died over 100 days at the hands of Hutu extremists in the 1994 genocide.