Neil Lennon's side, who overcame Ross County 4-1 in their last league fixture prior to the international break, were pegged back when former Celtic winger Niall McGinn scrambled home to cancel out Kris Commons' 36th-minute effort.
Derk Boerrigter fired home in the 91st minute shortly before Commons clinched his second as the defending champions left it late at Celtic Park.
Second-placed Inverness Caledonian Thistle beat St Johnstone 1-0, with Billy McKay notching up his 10th league goal of the season in the fourth minute.
Motherwell and Dundee United, in third and fourth respectively, were also victorious on Saturday against Kilmarnock and Partick Thistle.
Alan Archibald's nine-man Partick lost 4-1 at Dundee United. Gary Mackay-Steven broke the deadlock in just the second minute only for Partick to draw level 12 minutes after half-time as Aaron Muirhead netted a penalty.
Andrew Robertson put Dundee ahead again in the 70th minute before Conrad Balatoni was shown a second yellow. Mackay-Steven and Brian Graham netted before Stuart Bannigan also saw red.
Motherwell, meanwhile, who beat Allan Johnston's Kilmarnock 2-0 following second-half goals from John Sutton and Henri Anier, the former putting the visitors ahead with a diving header.
Elsewhere, the points were shared at both Tynecastle and St Mirren Park as Hearts netted a late equaliser through Ryan Stevenson to draw 2-2 against Ross County, while St Mirren and Hibernian played out a goalless stalemate.