"You could not arbitrarily take away someone's property rights simply because you wanted to (ie. you could not enter someone's house without reason and remove their guns or other property because you think that guns are "bad.")"
The mayor of New Orleans was finally convinced that was true by a federal court. By then the property was damaged beyond repair. If Lincoln had acted so arbitrarily in freeing slaves of loyal citizens, would they have avoided or delaid compensation so long as to make it not worthwhile, or offered no compensation at all? Stan