Absolutely you should have copies of this information. MRB has a service record filed as Matt R. Bane with Co. "F", 4th Confederate Regiment, which includes his prisoner-of-war record at Camp Randall (Madison WI), Camp Douglas (Chicago IL) and release at Vicksburg MS. The record states that he was among a group of 726 prisoners on board the steamer John H. Done who were released at Vicksburg on Sept. 20, 1862. He is noted as "sick" at the time of release.
The other record has been filed as Matthew R. Barnes, 54th Alabama Regiment. A note on the file jacket shows that the War Department clerk who compiled this information matched this service record with MRB's record with the 4th Confederate Regiment. You can order both records from the message board, as offered in the box seen above. There isn't a lot in either file, but it's the sum total of wartime data available for MRB.
Let me suggest that you post an inquiry for MRB on the Mississippi message board, asking for confirmation that he served in the 37th Mississippi Regt. You might also contact the state archives in Jackson MS and ask if their staff can locate a service file for the three-year period 1862-1865. Let us know what you find.
A final note -- the oath of allegiance on record for MRB does not include its actual wording. An internet search for the terms "oath of allegiance" and "north of the Ohio River" will produce examples of the terms required for Confederates taking the oath.