The Chelsea manager entered the match with just two points from his previous three visits to Newcastle, and watched on as his side wasted a chance to move to the top of the Premier League table on Saturday.
Chelsea controlled a first half devoid of clear-cut chances – their best effort saw John Terry hit the crossbar with a header – but were left reeling when Gouffran headed in an inviting Yohan Cabaye free-kick in the 68th minute.
Substitutes Willian and Samuel Eto'o went close for Chelsea in the latter stages, but Remy hammered home Vurnon Anita's cut-back with two minutes left to seal Newcastle's win.
The loss was Chelsea's second of the league campaign, which sees them fall five points adrift of London rivals Arsenal.
Arsene Wenger's men had gone into the game on the back of a League Cup defeat to Chelsea, but they bounced back to open up a healthy advantage over the chasing pack with a 2-0 win at Emirates Stadium.
Santi Cazorla opened the scoring as he pounced on a rebound after heading Bacary Sagna's right-wing cross against the post following a fine counter-attack from the hosts.
Liverpool had spells of pressure at various points in the match, but Arsenal killed things off after 59 minutes when Aaron Ramsey collected a square pass from Mesut Ozil on the edge of the area before unleashing a magnificent volley to all but the end the game as a contest.
Manchester City jumped up to fourth on the table thanks to a 7-0 drubbing of Norwich at Etihad Stadium.
A Bradley Johnson own goal set the ball rolling, before further first-half efforts from David Silva, Matija Nastasic and Alvaro Negredo put the hosts firmly in control.
Yaya Toure, Sergio Aguero and Edin Dzeko added goals five, six and seven after the break as the match lived up to its goal-rich reputation, with 25 having been scored in the previous four top-flight meetings between the two sides.
Meanwhile, defending champions Manchester United extended their unbeaten run to four games with a 3-1 triumph at Fulham.
Antonio Valencia was handed the easy task of tapping in from close range after Wayne Rooney had cleverly sucked in goalkeeper Maarten Stekelenburg.
Robin van Persie doubled the lead as he latched on to a ball from Januzaj before Rooney made it three goals in the opening 45 minutes.
Alexander Kacaniklic's deflected strike gave Fulham hope midway through the second half, but that proved to be nothing more than a consolation.
In other results on Saturday, Hull City claimed their fourth win of the season as they downed nine-man Sunderland 1-0, Stoke goalkeeper Asmir Begovic scored in the club’s 1-1 draw against Southampton, West Brom defeated manager-less Crystal Palace 2-0, while West Ham and Aston Villa played out a goalless draw.