Sensitivity towards race should be apparent in General Maury's request that a single company of Creoles be allowed into Confederate service. I have also already mentioned Alabama Confederates who were discharged on account of race. These were men who had served in their companies for some time, but were sent home due to questions about their ancestry. Remember that there were serious reservations in the North about arming black men. Race was a big deal on both sides of the Mason-Dixon line.
Try to find examples of slaves or FMoC enrolling in Northern regiments (not U.S.C.T.). At the time of the Civil War, it just wasn't done.
There might have been occasions in which trusted body servants might have picked up arms and participated in an engagement. Just my opinion - can't prove it one way or another. That may be what you had in mind as well.
The scene described in Frederick MD during Lee's first campaign north of the Potomac sounds reasonable. Except for the part about black men being armed to the teeth, of course. It could be that from a distance a Northern observer might mistake dark-skinned Confederate veterans with black men. It could be easier to make that mistake with body servants, teamsters and cooks moving along with the column of soldiers.
Whether we agree about this or not, it's nice to have a civil discussion.