Harry Clifton and Callum Jennings were on target in a 3-2 defeat at The Our Physio Stadium.
The hosts had held a first half the lead right up until the 75th minute when Harry Clifton scored, before going ahead through Jennings. But the home side weren't done and a rasping free-kick looked to have sent the game into extra time. But the Yorkshire side had other ideas and duly snatched the winner right at the death.
Albion will now face Wrexham or Abbey Hey in the 3rd Qualifying Round of the competition.
Alan Lamb's boys are next in action on Saturday when they face Chesterfield at Cheapside in a Youth Alliance League fixture (11am).
Town: Jack Pearson, Adam Lawis, Joe Crowther, Towm Sawyer, Jack Keeble, Cameraon Hough, Ben Sibbick, Harry Clifton, James Wroot, Callum Jennings, Kris Fawcett, Charlie Loft, Oakley Heath-Drury, Joey Finnegan, Lucas Middleton, James Phillips.