Since you live close to NARA, it may be worthwhile to view his service records for the hard proof you need.
One example would be his parole record at Vicksburg. He is listed as C.F. Fonby in the index and may have signed using a full name.
A sample can be found at this website, Capt. Gresham's initials are used at the top, but he signed Jas. F. Gresham on the bottom line.