James Larkin Shelby enlisted in Co. D, 2nd Battalion Arkansas Cavalry, at Pine Bluff, Arkansas, on March 1, 1862, and was mustered into Confederate service for twelve months by Maj. T. J. Reid. Effective May 15, 1862, the 2nd and 6th Battalions Arkansas Cavalry were consolidated to form the 2nd Regiment Arkansas Cavalry, and Shelby's company became (old) Co. G [subsequently redesignated (new) Co. F on August 12, 1862] at which time their term of service was extended to three years or the war. Shelby took the oath of allegiance to the United States at Pine Bluff, Arkansas, on March 4, 1864. A descriptive roll described him as a farmer, age 24, born in McNairy county, Tennessee, residence Jefferson county, Arkansas, eyes gray, hair auburn, complexion fair, height 5’7”.

After taking the oath of allegiance he joined the Union army. He enlisted in Co. K, 4th Arkansas Cavalry, U.S. Volunteers, at Little Rock, Arkansas, on April 24, 1864, for a term of service of three years. He was mustered out with his company at Little Rock, Arkansas, on June 30, 1865. An index of postwar U.S. pension applications states that he applied for a pension in 1883, and his widow applied for a pension in 1893.