Common mathamatics must at some point play a part in this discussion. No demongraphic (ethinic or otherwise) has ever has ever in history been 100% polarized on any issue because of the effects on individual lives.
According to the 1860 census there were 4,000,000 negros (free and slave) in the southern state that seceeded. Many of the free Blacks were skilled labor which had a strong tie to the communities that they lived in. For example the first person killed in John Browns' Harpers Ferry Raid was the black telegraph operator to keep him from sounding the alarm. Ironic isn't it?
In Louisiana you were considered to be of ethinic african origins if one of your greatgrandparents were of african decent. You were a Quadroon.
If only 1% of those of the ethinic african decent population served in any manner of Confederate Service that would be 40,000 men. Everybody seems to so hung up on the enlisted service and seem to excuse the nonenlisted service as slaves being forced to serve by their master. But in reading the period accounts in regimental histories of negro service, letters, the speeches of Frederick Douglas, who lamented the service of such Blacks in the Confederate Army, and even within the O.R's, in Confederate service it does not seem that such service was entirely involuntary. I have not read any accounts of Guards being placed on these servants. And nobody could be spared to chase them down if they did.
In reading, a regimental history of the 11th Mississippi Infantry comments on the service of 12 such servants that serve that regiment and how they (the servants) wepted at the Battle of Gettysburg because of their loss of so many of those they served. And that is not the only such account. The oppertunity for such "servants" to excape during any of the Army of Northern Virginias campaigns and camps would have certainly been great. And certainly would have been greatest during the Maryland and Gettysburg Campaign for example, since these servants duties would have been to gather food for an Army on the move.
In order to justify the rational for the conduct of a war you have to destroy the evidence which is contray to that purpose.