Dany N’Guessan’s double strike was in vain as the Valiants lost a seven goal FA Cup thriller at Vale Park.
Rob Page named an unchanged starting eleven with skipper Carl Dickinson on the bench, but it was the visitors who started the brighter with one effort hitting the bar.
However, Vale took the lead against the run of play on eleven minutes when Ben Williamson appeared to score although it could have been an own-goal.
But the visitors went ahead with a double strike. First, top scorer Afobe equalised on the half-hour mark and then Baker put them into the lead on 40 minutes. But Vale equalised just before half-time with N’Guessan’s first goal for the club.
Milton Keynes stretched their lead with a 59th minute penalty after Williamson fouled in the box and when Green made it 4-2 it looked all over.
But Mark Marshall’s shot rattled the woodwork before N’Guessan made sure and it was 4-3 with five minutes left.
Vale pressed for an equaliser but to no avail as they exited the cup at the first round stage.
Vale: Neal, Yates (Dodds), Duffy, Streete, McGivern, Birchall (Moore), Lines, O’Connor, Marshall, Williamson (Daniel), N’Guessan
Subs: Johnson, Dickinson, Brown, Slew