Fields, a victor in the time trial in 2012 and 2013, could only finish down in seventh after the American set a mark of 25.194 seconds.
Instead it was Willoughby, the 2012 men's elite world champion and Olympic silver medalist, that set the pace with a blistering time of 24.757secs.
The Australian took the title ahead of Fields' compatriot Corben Sharrah (24.819secs), with France's Joris Daudet (24.924secs) taking the final place on the podium.
That meant disappointment for reigning men's elite champion Liam Phillips and Olympic title holder Maris Strombergs, who finished fourth and fifth respectively.
Phillips came agonisingly close to bronze, the Briton missing out by just over nine hundredths of a second, with Strombergs three hundredths further back.
Meanwhile, there was home success for the Dutch fans to cheer on Saturday as Laura Smulders emerged victorious in the women's time trial.
Smulders, bronze medalist at the 2012 Olympics, ensured glory by crossing the line in 27.449secs.
The 20-year-old was first ahead of Caroline Buchanan (27.471secs), who sealed silver on a very successful day for Australian riders.
Defending women's elite champion Buchanan beat Olympic gold medalist and defending time trial champion Mariana Pajon to second in another tightly contested battle.
Pajon's bronze came thanks to a time of 27.553secs and she will now look to atone for the disappointment of failing to defend her crown by reclaiming the women's elite title she won in 2011.
The winners of the elite races will be decided on Sunday in the final day of competition at the Ahoy Arena.