By typical planting methods, it took about two decades for cotton culture to wear out the soil. Since the invention of the cotton gin, most Southern men expected to move when fresh land opened up somewhere else. That's why the issue of slavery in the territories split the Democratic Party in 1860. Citizens of the slave states didn't particularly appreciate being unwelcome in any part of the country.
If they had read Lincoln's speeches in 1859 more carefully, it's evident from his language that Northerners did not want the presence of black people in the western lands. IMHO much of the opposition to slavery was based on what we would call outright racism today. I have quoted Lincoln's remarks recently in which he mentions (slave n-word) as undesirable to white labor in the west.