The Bank of Sierra Leone announced the move last August, insisting that the public's purchasing power would not be affected by the change.
A note of 10 new leones will have the same value as a note of 10,000 old leones, the equivalent of just under one US dollar ($0.75).
"We are removing the 'zeros of shame' to get the currency properly aligned," Morlai Bangura, a central bank director, told AFP.
He said the bank had begun distributing the new notes to commercial banks last week.
Year-on-year inflation in the West African country was 24.87 percent in May, according to country's statistics agency.
Rising prices have driven the printing of banknotes, resulting in a mountain of paper money that is costly to sustain and unwieldly for the public.
Shoppers need huge quantities of banknotes for the simplest transactions, and unscrupulous bank tellers sometimes pilfer notes out of sealed bundles of bills.
"Our economy is not doing well -- the cost of rice, flour and sugar are going up daily," said Abubakr Kamara, a restaurant owner in Central Freetown, at the time of the announcement.
Sierra Leone's eight million people live in one of the poorest nations in the world, ranking 182 out of 189 countries in the UN's Human Development Index.
Its economy, heavily dependent on minerals, was devastated by a civil war that ran from 1991-2002 and left about 120,000 dead.
Efforts at rebuilding were set back by an Ebola epidemic in 2014-2016, a fall in world commodity prices and the coronavirus epidemic -- all of which have disrupted trade and investment and hit exports.
Sierra Leoneans will be able to use both the old and new notes during a transition period until September 30.
From October 1, the old currency will cease to be legal tender.
The public will be able to swap the old currency for the new one until November 15, Kallon said in a statement.