Although not a contemporary source, "The Encyclopedia of Civil War Usage" by Webb Garrison may be helpful.

It describes an "Affair" as an engagement of minor size or consequence.

An "Action" is a widely used but loosely defined term for an engagement between enemy forces; often treated as synonymous with "Battle".

A "Skirmish" was light combat that involved relatively few men. In DYER'S COMPENDIUM an attempt was made to categorize every military event according to sixteen varieties. In this work, a "Skirmish" was larger than an "Operation" but smaller than an "Assault".