After withstanding pressure for long periods of the first half, the Mariners took the lead against the run of play through Jordan Cook.
And they defended well after half-time to keep their visitors at bay and claim a deserved place in the next round, where they will face Macclesfield Town.
Town boss Michael Jolley made seven alterations to the starting XI after Saturday's draw with Bradford City, with only James McKeown, Luke Hendrie, Harry Davis and Elliott Hewitt remaining in the line-up as others were given their opportunity to impress.
Town: McKeown, Hendrie, Pollock, Davis, Gibson, Hewitt, Ring, Cook, Wright, Vernam, Ogbu
Subs: Russell, Hope, Hessenthaler, McPherson, Painter, Green, Cardwell.
Rovers: Seny Timothy Dieng, Brad Halliday, Reece James , Tom Anderson, Cameron John, Benjamin Whiteman (C), Alex Kiwomya, Matty Blair, Ali Crawford, Kieran Sadlier, Niall Ennis.
Subs: Ian Lawlor, Shane Blaney, Madger Gomes, Ben Sheaf, Jon Taylor, Will Longbottom, Alfie May.