The diplomat and two other arrested people are suspected of plotting to attack a meeting of exiled opposition members in Paris last weekend.
Tehran denies any involvement.
The diplomatic tension comes a day before President Hassan Rouhani is due to arrive in Austria to discuss a disputed nuclear deal.
Two Belgian nationals apprehended in Brussels are reported to be husband and wife, and to be of Iranian origin.
The Iranian diplomat, identified as Assadollah A, is ordinarily based in the Austrian capital. But he was arrested in Germany.
Iranian foreign ministry spokesman Bahram Qassemi told reporters that his country had evidence that the attack had been planned by a terrorist organisation.
“The latest scenario has been planned and carried out to damage Iran-Europe ties amid such sensitive and crucial times,” he said.
What was the alleged bomb plot?
Supporters of Iran’s opposition had gathered in Paris for a rally of the National Council of Resistance of Iran (NCRI).
The group portrays itself as a parliament-in-exile, made up of a wide array of disparate groups. It says “tens of thousands” attended the weekend’s rally at the Paris Convention Centre.
On Saturday, the day of the rally, the two Belgian nationals were caught by Belgian police, with half a kilogram (1.1lb) of homemade explosives along with a detonator, foiling the plot.
The arrests were revealed on Monday when the pair were partly named as Amir and Nasimeh, along with their ages – 38 and 33.
The Iranian diplomat Assadollah A, meanwhile, was arrested by German police. The NCRI alleged he had been the head of intelligence at the Vienna embassy since 2014.
The NCRI accused the Iranian government of being behind the attack.
Tehran, however, accused the group’s own members of staging an attack on itself to damage Iran’s attempts at salvaging its nuclear deal.
Donald Trump withdrew the United States from the landmark sanctions relief deal in May, despite protests by his European allies.
Iranian officials are now attempting to convince the other signatories – China, France, Germany, the EU, Russia, and the UK – to remain committed to the deal.
What happens next?
Assadollah A is in the hands of German police, and a European arrest warrant has been issued for him.
Austria has asked Iran to lift his diplomatic status, officials at the foreign ministry said, but Austria can do so itself, and will do so within 48 hours.
But lifting diplomatic status will have no immediate effect. In Germany, he is not accredited as a diplomat – and his immunity would only come in to play if he were to be extradited to Austria.
Any intention to request his extradition has yet to be announced.
Meanwhile, President Rouhani is due to arrive in Austria as planned on Wednesday.
The landmark nuclear deal opposed by President Trump eased economic sanctions on Iran in return for reducing its nuclear capabilities and submitting to inspections.
President Trump labelled the deal “one of the worst and most one-sided transactions” his country ever signed. Iran, meanwhile, has increased its ability to enrich uranium once more – and said it would continue the process if the deal collapsed.
To date, European powers have broadly supported continuing with the deal despite the absence of the US.