Mr Brown --
I cannot agree with you. This man took the oath of allegiance on Aug 22, 1864. He wouldn't have joined a Confederate cavalry unit earlier because that would be desertion. He wouldn't have joined later because he had taken the oath to the US.
The record does show that this man was a resident of Green County MS. Why would be be buried at Chattanooga?