The “Swift” Mullins family appear in the 1860 Franklin County, Arkansas, census. His wife, Caroline, and a son, John, age 18, were in the household, as well as John’s siblings: Mahala, age 15, William F, age 12, Elizabeth, age 11, Julia Ann, age 9, Isaac, age 5, Martha, age 2, and Benjamin, age 1. This looks like the family you described, and if so, Company G, 35th Arkansas Infantry, came from Franklin County.

The Gerdes site reveals that a Private John Mullen, CO G, 35th AR INF, enlisted 18 JUN 1862 at Van Buren, Crawford Co., AR (next county west of Franklin). The age and state of nativity do not match your John, however, but this is not a rare occurrence when dealing with census records. Perhaps this information will give you a place to start. Good luck in your research.

Source:
Swift Mullins household, 1860 U.S. Census White Oak TWP, Franklin County, Arkansas Population Schedule,Page 48 (reverse of stamped page 287), Lines 27-36, Dwelling 336, Family 324, National Archives micropublication M653, Roll 41