The hosts had looked set to claim a miraculous draw, entering the final five overs with two wickets still remaining, but Harris bowled both Dale Steyn and Morne Morkel to clinch a 2-1 series win and ensure Graeme Smith's last international match ended in defeat.
MORE: Photo Gallery South Africa vs Australia 3rd Test
Resuming on 4-71 with AB de Villiers and nightwatchman Kyle Abbott at the crease, South Africa were required to bat through 98 overs on Wednesday to secure the draw. And for long periods, it looked as though they would manage it.
With a victory target of 511 out of the question, it was all about batting time for the South Africans and Abbott lasted 89 balls in total for just seven runs before leaving a delivery from James Pattinson (2-62) that crashed into off stump.
De Villiers carried on manfully, continuing to frustrate the Australian bowling attack with stoic defence, before Harris removed him with the new ball, caught behind with an excellent delivery for 43 off 228 deliveries in over five hours.
Faf du Plessis, who made and unbeaten 110 at Adelaide in November 2012 to deny the Australians in similar circumstances, was next to fall for 47, lbw to Steve Smith (1-43), but not before eating up 18 overs with JP Duminy.
Duminy and Vernon Philander also hung around, adding 73 runs in 17 overs before the former was caught at leg slip by Nathan Lyon for 43 off the bowling of Mitchell Johnson (3-92).
With only Steyn and Morkel to come and 20 overs still remaining at that point, all looked lost for South Africa, but Philander and Steyn survived 15 of those, the only scare coming when Philander was originally given out caught at short leg off Johnson.
South Africa reviewed the decision and, although it appeared the ball had brushed his thumb before crashing into the helmet and into the arms of the fielder, replays showed his hand to be off the handle and so the decision was reversed.
That decision sparked a confrontation between Michael Clarke and Steyn, with umpire Aleem Dar stepping in to calm tempers.
Steyn and Philander seemed unshaken by the near miss.
Philander went on to make an unbeaten 51, but Clarke called on Harris with five overs remaining and the 34-year-old removed both Steyn (one) and new man Morkel (zero) with full, straight deliveries to finish with figures of 4-32 and clinch a dramatic victory for his side.