There are a number of cases of people like you describe who sided with Louisiana and the Confederacy. In other cases, some individuals remained loyal to the Union.