For what it's worth, the most of the U. S. 1890 census returns were destroyed in a fire, but 1880 and 1900 census returns should help. Household members are reported back to 1850 -- head of household only prior to that.

The first U. S. census was taken in 1790. Your local library reference department or archives will provide information as to which ones are available. If you don't have access to a library that has the recors you need, you might consider how a service like Ancestry might help.

Here's a good starting point for earlier Georgia census records. http://www.mygeorgiagenealogy.com/ga_maps/ga_cm.htm

Keep in mind that new counties were created out of old ones. For instance, you may have an ancestor who lived in McDuffie County in 1880. They might appear in the 1860 census for Columbia County and still live in the same house because McDuffie was created from part of Columbia.

Your grandfather's death certificate should provide names of his parents and their birthplaces. It should also state where he was born. You can also look at the census nearest his birthdate (1910, 1920 &c), get his parents marriage certificate and other documentary evidence.

At some point you may be obliged to take your best guess on a family relation, but do this only when you have no other alternative. Feel free to ask questions about Confederate military records or military service here.