It appears that the Jeremiah Gibson you seek served first in the 34th Battalion Virginia Cavalry, then later in the 22nd Virginia Cavalry.

He will have 2 Compiled Service Records, which can be ordered at: http://history-sites.com/research/index.html

The following volumes of the Virginia Regimental Histories Series will be helpful.

34th Battalion Virginia Cavalry: http://www.worldcat.org/oclc/29381657&referer=brief_results

22nd Virginia Cavalry: http://www.worldcat.org/oclc/26035418&referer=brief_results

Family files at Ancestry.com indicate that Jeremiah "Jerry" Gibson was born in October 1838, lived in McDowell County, [West] Virginia in 1860, and later lived in Putnam County, West Virginia. You should be able to access this info at a good public library.