Rovers made the early running but Harwich battled hard to keep them at bay and took the lead, following a smart breakaway and an opportunist finish by Jack Milne (pictured).
With the Harwich team having Colchester connections, there was a local-derby flavour to the match and a large enthusiastic crowd in attendance.
With regular Harwich goalkeeper Oscar Finbow still making his way to the ground, Rueben Shaw stepped into the breach for the first half and gave an excellent performance, pulling off some smart saves.
Rovers were enjoying a good share of the attacking play but Harwich's breaks were dangerous and Dylan Crowley increased their lead with another well-taken goal.
With half time approaching, Rovers were rewarded with a shot that gave Shaw no chance.
Rovers started the second half strongly and deservedly equalised. With their tails up, they soon took the lead.
Will Crook raised Harwich hopes with a long-range free kick which struck the post and, in the closing stages, the Shrimpers staged a strong rally with Liam Clarke's effort hitting the crossbar.