That is the question and problem with the Confederate Army organization, it was too rigid in its assignment of men and materials. As far as the War Department in Richmond was concerned these men, although geographically in Virginia, belonged to the Department of Trans-Mississippi and were ordered to return. We today think too liberially of how men are assigned to units, just because they were captured in Arkansas and transported to City Point, Va. they were still assigned to a department West of the Mississippi and had to be returned. As were the Confederate soldiers captured at Fort Donelson, they were transported North, and when exchanged, were sent back to their original department and aided in the repluse of Grant in North Central Mississippi. The Confederate War Department was too anal in this respect and it led to difficulties in defending a very large interior line.
I agree. Richmond should have assigned troops where they were most needed, but that is not how they were handled administratively. Many Confederate unit enlistments had geographical agreements. Militias, Home Guard, Concripts, Volunteers, Provisional, and Regulars all had their restrictions by law under who and where they served.
It was that way after the Civil War too. Exchanged or repatriated prisoners were almost always returned to their orginial commands.