Note: The below information does not match up perfectly but I am including all of it because I believe you should follow up on both. I confess I am not too good good at geneaology research. If he was indeed discharged APR 26, 1865 he must have enlisted in another unit or perhaps I missed some of his service record somewhere.

Source: Mississippi Confederate Graves Registry
http://mscgr.homestead.com/

Roberts, Henry
Capt; Co. A, 24th MS Inf
Born: Aug 15, 1822 Kittrell, MS.
Died: May 17th 1880 MCClain, MS.
Discharged: Apr 26, 1865
Cemetery: Breland, McClain
____________________________

Source: Various rootsweb entries.

Henry ROBERTS
Birth: 28 MAY 1822 in Leakesville, Greene, Mississippi
Death: 5 JUL 1888 in Greene County, Mississippi 1

Father: Isaac ROBERTS
Mother: Elvira CHATHAM

Marriage 1 Priscilla Jane COCHRAN b: 22 FEB 1836 in New Augusta, Perry, Mississippi

Marriage 2 Louisa Jane BRELAND b: 5 DEC 1833 Birth: Dec. 5, 1834
Death: Oct. 11, 1868 (Buried Buffalo Cemetery, Greene County
Mississippi, USA (burial info from Find-a-Grave)
Married: 1851 in Greene County, Mississippi 1

Children
1. William Columbus ROBERTS b: 27 APR 1852
2. Catherine Alice Alabama ROBERTS b: 1854
3. C.C. ROBERTS b: 1856
4. Lorenzo Dow ROBERTS b: 26 FEB 1857
5. Thomas ROBERTS b: 1858
6. Miranda Arizona ROBERTS b: 14 MAY 1859 in Greene County, Mississippi
7. James ROBERTS b: 1860
8. Henry Davis ROBERTS b: 1861
9. Agnes E. ROBERTS b: 1864
10. John I. ROBERTS b: 1866
11. Louisa J. ROBERTS b: 15 AUG 1867

Buried: Old Buffalo Cemetery, Mcclain,Mississippi
____________________________________________