Southern soldiers were far superior in using a rifle and riding horseback to their Northerner counterparts in the Army of the Potomac. I guess the Union special forces you are speaking of would be the Berdan Sharpshooters. A regiment of well trained snipers. Hiram Berdan, Amos Jones, Rev. Lorenzo Barber etc. See this article, https://en.wikipedia.org/wiki/1st_United_States_Sharpshooters ...However the average Southern soldiers were far more skilled than his enemy, in my view.
Regards, James.