Your James Anderson can be found in the Calcasieu Parish (Imperial Calcasieu) US census for 1860 (Camerson Parish was not formed until after the war), and in Cameron Parish in 1870 and 1880. Based on this I suggest a few ideas you may try.

1. Check the Louisiana Death registries (New Orleans Library or any Parish courthouse), looking for a James Anderson who died in Cameron Parish after 1880, but probably before 1900.
2. Living in Cameron Parish he may have been Catholic so you can try to find Catholic church records for his birth (1812 to 1815) and his death and burial (1880-1900). The diocesan records still exist for much of South LA. If not Catholic, some of the local Protestant churches may also retain old records.
3. Check with the Southwestern Louisiana Genealogical Society in Lake Charles - what records do they have, or suggestions.
4. Check the archives and manuscript collection at the McNeese State University in Lake Charles.
5. Check records at the Cameron Parish Courthouse (assuming they survived Audrey, Rita and Ike).

I list these suggestions feeling that you have probably already done most of this, but I thought it good to list them.

I noted 18 James Andersons from Louisiana in the NPS-CWSS, and another 37 J. Andersons - 55 total possibilities. In Booth, I saw 19 James Andersons, but did not count the J. Andersons.

Good luck with your search.

Don Parker