First they weren't rich enough to even hope to own a slave. Second they didn't fight in many wars, historically most of them were preachers. Most were also frontiersmen opening new territory before it became profitable to own slaves. But when they did fight they were protecting their home. For the most part they were loyalist to authorities. One Grandparent was even kicked out of this country for being a Troy and fighting for King George III during the Revolution. And finally they were not conscripts as I said they were volunteers of 1861, which means they went when they were not required by law to do so. Which means that there was something in their core beliefs which motivated them to do so, and I can't find any record that that motivation had any association to do with the slave.