You are right. Blacks did enlist in various (predominately white) units from 1861-on. As musicians, cooks, teamsters, and privates.

I should have worded my post differently. 1864 was the first time a law called for their enlistment as a group.