Pamela --
Soldiers released under an exchange agreement did not take the oath of allegiance. They were released under 'promise' not to fight again until properly exchanged. The same was true for soldiers who surrendered at Vicksburg.
A soldier who took the oath of allegiance was released under promise to remain north of the Ohio River. Your choice was to remain north of the Ohio inside a prison camp, or enlist in a northern military unit.