Best I could find is this source:

http://fisher.lib.virginia.edu/collections/stats/histcensus/

It can get you as far down as Orleans Parish for data. I tried to see if there were anything on the US Census site, could not find any.

JIM T