If you have not done so, find him in the census. Start with the 1860 Census. This should show where he was born.
Grinnell was in Company E "Wheat's Life Guards". This was formed primarily in Orleans Parrish. Start with the Orleans Parish census of 1860.