Dunne, a popular figure at Colchester following two spells at the Essex club as a player, was appointed as John Ward's successor in September 2012.
He duly oversaw finishes of 20th and 16th in England's third tier, but a poor start to the new campaign has led to Dunne departing the Weston Homes Community Stadium.
Colchester are second-bottom in League One, having lost four games in a row following their opening-day draw against Oldham Athletic.
A club statement on Monday read: "Joe Dunne has left his role as first team manager of Colchester United FC by mutual consent.
"Joe has managed Colchester for two years and has successfully kept the club in League One, despite the large cuts the club made to its playing budget following the introduction of the Financial Fair Play rules that clubs have to abide by."
Colchester chairman Robbie Cowling added: "Joe has done a remarkable job as manager for Colchester United over the past two seasons and he has been a remarkable servant for the club for many years.
"He is a Colchester United legend and I am grateful for the way he kept our club in League One during these challenging times.
"I wish him all of the very best for the future."