One other point, which Miles alluded to briefly in another post. State laws north and south of the Mason-Dixon line as they existed in 1860 often didn't allow recently-freed slaves to remain inside state lines. A freed slave had six months to leave. So why were there so many free blacks in some Southern cities? Because the laws usually stipulated that freemen who were descended from free residents during the Spanish or French colonial era were exempt. Legislation was explicitly designed to keep recently freed slaves from becoming a burden or nuisance in their communities.
This was true in Alabama and Mississippi - it was also true in Indiana and Ilinois.
Had there have been time to pursue the Black Confederate experiment, slaves freed by their masters to become Confederate soldiers would be escorted beyond state lines once they got out of the army. State laws usually punished those who tried to return with imprisonment, flogging and being returned to slavery. That was part of the Alabama legal code in 1860, and probably in the laws of other states.
Ya'll come back now, ya hear?
You don't really sing that song, do you? Tell me it ain't so.