I would guess the reason is a little deeper than that. Most of the men who came home were in ill health, families were shattered, the cities, mills and what industries, railroads roads, bridges etc., we had had been destroyed. Widows could not afford to hire help, black or white so taxes took the land. Reconstruction and the carpetbaggers got their share. In reality we did more to rebuild our enemies after WWII than what was done to rebuild the South. The North wasn't even scratched.
GP