If you have time you should vist Port Gibson. Old dirt roads through the area of the battle will put you back in 1863. The A. K. Shaifer House can be visited along with other spots. Visit http://civilwartraveler.bravepages.com/GrdGlf/GrandGulf.html and get some ideas.

In addition there are some gorgeous antebellum homes in town.