Finlay got on the end of Justin Meram's long pass from half-way in the 84th minute and stabbed his shot past New England's Bobby Shuttleworth to secure three points for Columbus, which took the Crew level with Toronto FC in the Eastern Conference.
Federico Higuain had given Columbus the lead just before half-time at New England's Gillette Stadium with a pin-point free-kick and, although the hosts equalised soon after the break through A.J. Soares, Finlay's fifth goal in 10 games saw the visitors triumph.
Columbus moved to 26 points for the season, level with Toronto, who lost 2-1 at home to Sporting Kansas City, although the Canadians remain ahead of the Crew on their number of wins.
Sporting remain top in the East with 38 points after their victory in Toronto with DC United (34 points) in second.
Toronto (26 points and seven wins) sit just ahead of Columbus (26 and six); despite having only won one of their past seven MLS fixtures.
Higuain scored the goal of the day in MLS on Saturday, curling a 20-yard free-kick over the wall and just underneath the crossbar to put the Crew 1-0 up in the 43rd minute.
Soares dragged New England level in the 50th minute, heading home at the near post from Kelyn Rowe's corner but in the closing stages, Columbus snatched the win through Finlay.
At Toronto's BMO Field, the home side opened the scoring in the 16th minute with Dominic Oduro exhibiting brilliant vision to pick out Jackson Goncalves on a diagonal run and the Brazilian striker rounded Sporting goalkeeper Andy Gruenebaum to convert.
But the visitors fought back in the second half with Graham Zusi curling a shot into the net from the edge of the box two minutes after the break, while Jacob Peterson completed KC's comeback in the 80th minute, firing low past Toronto keeper Joseph Bendik after a back-heel from Dom Dwyer.
The victory, which came despite losing Matt Besler to a red card in the 75th minute, was Sporting's fourth in a row in the league, as the reigning champions continue to surge after a poor run through May and June where they failed to win in five games.