AskHistorians

Large-scale farming wasnt profitable in the North.

I'd like to point out that large-scale farming was very profitable in the North - just not of cotton and tobacco. Northern grain was as important an export as Southern cotton - but grains are generally more land-intensive, while tobacco and especially cotton are more labor-intensive (thus making use of slave labor profitable).

You can also see the difference you mentioned between the majority of the South and Appalachia, which remained a stronghold of Unionist sentiment. Appalachia is terrible country for massive cotton plantations, so slaver power never really set down deep roots there.