Police and intelligence officers searched three properties belonging to Bunyoni on Monday, but found no trace of him, according to security sources and media reports.
"They are still looking for him," Interior Minister Martin Niteretse told a press conference, adding he did not know the reasons behind the operation.
The former prime minister "had been warned in advance that the noose was going to tighten on him, he disappeared into the wild before the arrival of the police," a senior military official told AFP on condition of anonymity.
The official said the authorities had arrested a senior police officer for allegedly telling Bunyoni about the search operations.
Bunyoni, once police chief and security minister, was fired last September in the first major reshuffle at the top of Burundi's government since President Evariste Ndayishimiye took office in 2020.
An influential senior figure in the ruling CNDD-FDD party who was appointed prime minister in 2020, Bunyoni was sacked days after Ndayishimiye warned of a "coup" plot against him.
Bunyoni was seen as the leader of the "hardliners" among the generals who wield true political power in Burundi, with Ndayishimiye himself alluding to his isolation in a 2021 speech.
Ndayishimiye took power in June 2020 after his predecessor Pierre Nkurunziza died of what the Burundian authorities said was heart failure amid widespread speculation he succumbed to Covid-19.
In 2015, Nkurunziza oversaw a crackdown on political opponents that left 1,200 people dead and made Burundi a global pariah.
The turmoil erupted after he had launched a bid for a third term in office, a move the opposition said was unconstitutional and violated a peace deal that ended a bloody civil war in 2006.
The United States and the European Union had imposed sanctions over the unrest that also drove 400,000 people to flee abroad, with reports of arbitrary arrests, torture, killings and enforced disappearances.