That is true. My statement did leave a lot of variables, such as protection by the federal army and laws, also how they would be welcomed in the new territories and the North etc. Still I would think a good many would try freedom and some of these would still end up working for the same overseers they had worked with before.
GP