Mulaudzi took 800m gold at the 2009 World Championships in Berlin, five years on from claiming silver in the same event at the 2004 Olympic Games.
Ahead of his podium finish in Athens, Mulaudzi was his nation's flag bearer at the opening ceremony.
On Saturday, IOC president Thomas Bach said in a statement: "Our sympathies go to his family and friends, and to the whole sports family of the rainbow nation of South Africa.
"Mbulaeni Mulaudzi was a true Olympian and a fine athlete. A world champion, he always showed huge commitment to his craft.
"He proudly represented his new nation around the world, and at home his successes inspired a new generation of young people to take up sport. He will be missed by the Olympic family."
Mulaudzi was also a gold medalist at the 2002 Commonwealth Games and the 2004 World Indoor Championships, with a World Championship bronze coming in between those successes in 2003.
He returned to claim silver at the World Indoors in 2006 and 2008 on the way to his career highlight in Berlin, a fortnight after which he set a personal best of one minute, 42.86 seconds at a meeting in Italy.
Mulaudzi died aged 34.