Grondona, who had been in charge of the organisation since 1979, was reportedly taken to hospital on Wednesday after suffering a cardiac arrest.
His death was subsequently announced by the South American Football Confederation (CONMEBOL).
During Grondona's tenure in charge of the AFA, Argentina reached three World Cup finals, winning the first of those in 1986 before West Germany gained revenge to lift the trophy four years later.
Earlier this month in Brazil they reached the 2014 showpiece, only to be beaten by Mario Gotze's extra-time strike as Germany prevailed again.
Grondona's long career in football had also seen him serve as FIFA's senior vice president and as a member of the world governing body's Executive Committee.
FIFA president Sepp Blatter led the tributes to his former colleague, tweeting: "[I'm] very sad for the loss of a great friend.
"Julio Grondona left us at the age of 82. Today embrace his family. He rests in peace."
Lionel Messi also expressed his grief at the news, posting on his official Facebook page: "Really sad day for the football community, for Argentina and for me.
"Our AFA President, Julio Grondona, has left us. I want to send my condolences to the family and friends of the beloved."