United manager Nigel Clough had played down the importance of making a strong start in the build-up to Saturday's season-opener, and it was just as well as the Bramall Lane outfit lost 2-1 to Steve Cotterill's men.
Aaron Wilbraham opened the scoring with a header in the 20th minute, before Michael Higdon pulled the hosts level - both players netting on debut for their respective clubs.
Wade Elliott - playing his first match as a permanent member of the Bristol City squad - scored the winner in the 72nd minute after being teed up by a delightful Wilbraham flick.
However, Clough was able to take positives from his side's display.
"The only consolation we take is that it was a pretty good performance at times for long periods," he told the club's website. "First half we played some great stuff and created enough chances in that period to have won the game, but we didn't take our chances.
"We're pleased generally with the way we played - the manner of the goals that we conceded was the biggest problem.
"With a little bit of luck today, we certainly wouldn't have lost the game and we could have gone on and won it as well."
Yeovil Town endured a difficult return to League One as they suffered a 3-0 home defeat at the hands of Doncaster Rovers - another club to have fallen through the Championship trap door at the end of last season.
And it was a grim day for the third of the relegated teams too, with Barnsley beaten 1-0 at home by Crawley Town. The hosts' Dale Jennings was shown a red card on the opening day for the second year in a row after reacting angrily to a challenge by Crawley's Gwion Edwards.
Two of the teams who have come the other way - having been promoted from League Two - enjoyed encouraging starts to the campaign.
Last term's fourth-tier champions Chesterfield claimed a 2-1 win at losing League One play-off finalists Leyton Orient, while Fleetwood Town beat Crewe Alexandra by the same scoreline.
Rochdale lost 1-0 at home to Peterborough United to join 10-man Scunthorpe United - 3-1 losers at Swindon Town - as one of two losing promoted sides.
Milton Keynes Dons took a while to come to life as they came from 2-0 down to win 4-2 against Gillingham, although they were aided by two own goals by the visitors to Stadium MK.
There were also plenty of goals at Valley Parade, where James Hanson scored in stoppage time to secure a 3-2 success for Bradford City against Coventry City, who had equalised in the 89th minute through Reda Johnson.
Elsehwere, Colchester United were held 2-2 at home by Oldham Athletic, while Walsall's visit to Port Vale and Preston North End's home fixture against Notts County both ended 1-1.