The general source I cited was Steven H. Newton, Lost for the Cause: The Confederate Army in 1864.
Both sources are using reports like one seen in The War of the Rebellion: A Compilation of the Official Records of the Union and Confederate Armies, Ser. 1 - Vol. 38 (Part III) , page 683, "Abstract from Inspection Report of the Light Artillery, Army of Tennessee, Gol. B. F. Beckham, Provisional Army, C. S., commanding, for September 20, 1864, Capt. Charles Swett, Mississippi Artillery, Acting Inspector-General."
I recall seeing reports submitted to the A&IGO during 1864 which listed free negroes conscripted in Virginia and assigned to various auxillary departments. These were part of M935, Inspection Reports and Related Records Received by the Inspection Branch in the Confederate Adjutant and Inspector General's Office, copied on microfilm by NARA.