"Excuse me, sir," you say, "we've heard rumors that we're standing against our King. We're not sure where our duty lies." (DM rolls 1d20+3: 15!) The knight looks startled. "Henslow militia certainly is an ethical bunch. And armed with swords too! Not paladins in disguise, are you? Ho ho! In this case, duties to Duke and King cancel each other out. Follow whoever rewards you best." He offers you a clinking purse.
Use distract attention ability to get everyone's attention. Yell to everyone "The traitorous Duke has been offered mercy and will withdraw. Consider if your life is worth a few coins and the shame of treason when the King arrives!"
We're not actually with the militia, everybody here should already be on our side (ideally), except the lost knight. Smile politely, accept the purse, and go with the knight to the front gate. That will be a good place to observe and influence what's coming. [Who is here, exactly?]
"A paladin must be a good man. We're loyal to our King despite having the choice. And we're not alone in fighting the Duke. If you care most about wealth, consider the fate of whoever sides with the losing Duke and the rewards the winning King will bestow."
You return to your camp and rest. Over breakfast, you tell your friends about your mission. Your sergeant says, "If that was Lady Angelia, the Duke will listen to her. Guess we won't be fighting today. Have a sausage." You're still eating when (DM rolls 1d12: 4!) a knight comes by. He says, "You the militia from Henslow? You're with me in front of the main gate. Get armed. The enemy will be here by noon."
"You first, sergeant!" Let's see who our enemy is supposed to be.
Let's obey his orders but try to position ourselves some place where we can easily disrupt the Duke's troops' lines.
Share our intelligence with the Knight before doing option 2.
Option 2 only. This is not the same knight we saw in the chapel.
By the way, is there a reason we're in front of the gate instead of in, on, or behind it? Shall we blunt our swords and take off our armor before the battle also? If every officer we have will sabotage our chances with the intensity and creativity of this knight, our cause is hopeless.