I note quite a few Galvanized Yankees who joined the U. S. Volunteers and shipped out west, deserted as fast as they could.

Appears to me, they may have crossed over just to escape imprisonment and had no intention of remaining a Yankee.