Some helpful sites:
This site does not have a B.S. Smith or a Benjamin S. Smith from Mississippi, however it does list 2 B. Smiths with no middle initial, 1 Ben Smith (NMI), and 9 Benjamin Smiths (NMI). It would take some work cross-referencing, but if you look at each man's unit on this site:
you might be able to find one of these men in a unit from Lafayette County. You can then order that man's records from this board by clicking the banner at the top of the page.
However, one important question should be answered, if possible, before beginning that search. You do not mention that he was on the 1860 census with the rest of his family. While there might be another legitmate reason for him not appearing there, the most likely reason is that he was already dead in 1860, and thus could not have served during the war. Sometime oral tradition passed on from family member to family member can change from the actual circumstances.
Good luck in your search!!