You must also consider the possibility that this man didn't enroll. As mentioned, James B. Smith (yes, even with the middle initial) is fairly common. Several men by that name could have enrolled in Confederate service.

If Smith did enroll, his name could be recorded as J. B. or simply James Smith. If he or his widow applied for a pension after the war, locating such a document would aide your search. Otherwise you will need to check each service file to look for mention of residence and age. This will not be easy.