I think it had more to do with how the Confederate Army was organized. Those men did not belong to Robert E. Lee's Department. They belonged to the Department of Trans-Mississippi. Lee was not the overall commander of the army, not until 1865. Each department was like its own little nation and had it's quota of troops and supplies, and they were very careful not to disturb that arrangement.