There are about 250 individual "colored" claims, most detailing service as body servants, teamsters and cooks. Based on birthdates alone (1848-50), many were clearly too young to have shouldered a musket.
Most states adopted pension legislation like that of the State of Alabama. It did not provide for "colored" applications, so a few were received them among the tens of thousands of applications submitted over the years. I don't recall the details, but an application by an elderly black men witnessed by two Confederate veterans did state that he was with an Alabama command in Virginia during the war. He did not claim to have been a soldier - just that he accompanied this particular regiment.
The pension board rejected the claim on the basis that the applicant was a black man and never received pay from the government for military service. The same kind of argument was used to reject claims by home guards and militiamen not in actual service.
Today he might have sued in court because pension legislation wasn't restricted on the basis of race. Then it was simply understood to be the case.