A lot of men in northwest Alabama wanted to remain neutral and not fight for either side. They were called "mossbacks" and hid out from Confederate conscription officers. Some of these were eventually caught and went ahead and wore a Confederate uniform. When the Confederacy started applying more heat in regard to conscription in 1863, some of these Alabama men made the trek to northeast Mississippi and signed up with the Union's 1st Alabama. Many brothers fought on opposite sides--not because of convictions but because one brother was conscripted by the Confederacy and the other came in contact with Union soldiers on recruiting missions and signed up.
My g-grandfather was forced at gunpoint to join Patterson's 5th Alabama Cavalry, subsequently deserted and then signed up for a three-year hitch in the Union's First Alabama Cavalry.