At the time of writing this letter, Bailey had seen no action and was only witness from behind the lines.
He is recording second hand knowledge and whatever he claims, I would suggest are nothing more than shear conjecture. Prisoners he believes are soldiers, were most likely impressed laborers.
I would suspect Yankee soldiers treated blacks harshly, but this rectital may be somewhat exaggerated.