"It is with regret that our diplomat [previously identified as Charles Dickens Imene Oliha] was involved in an alleged rape incident with... (a) New York City resident," the ministry said in a statement published on the government's official Twitter page late Thursday.
"The diplomat in question is now back in South Sudan and has been suspended from his duties, awaiting the outcome of this investigation," the statement said.
The government in Juba's announcement did not elaborate on the allegations, but US media reports said the diplomat was accused of forcibly entering the victim's Manhattan apartment and raping her on Sunday.
Police initially took him into custody before releasing him hours later after he invoked diplomatic immunity.
"Sexual misconduct in any shape or form is heinous and wholly unacceptable," South Sudan's government said, adding that "a specialized committee" was examining the case.
The US State Department said Wednesday that it was "aware of the incident... involving a diplomat accredited to the UN."
"We take these allegations very seriously and are working closely with the New York Police Department and the Mayor's Office of International Affairs," State Department spokesman Vedant Patel said, without offering further details.
In March, the UN accused members of South Sudan's government of committing human rights violations "amounting to war crimes" in the country's southwest, urging investigations against dozens of individuals, including for sexual violence and abuses against children.