Susie --

If this man was captured, records of imprisonment and release as you have surmised would be found in his military service file. Both regiments mentioned did serve in Mississippi during this period, so there may be a connection.

Copies of both complete service files are available here.