This is not from any official source, but I have drawn conclusions based on reading the old Union army official records of these things over a number of years. I would like to see input on this puzzle from others, as these are just educated guesses.
"Affair." Based on what I have seen, this refers to a shooting accident between two Union units mistaking each other for the enemy, or a serious error involving Union troops possibly shooting. I have also seen this title put on depredations or atrocities by Union troops. The only common factor I can see is that enemy combatants are usually not involved in "affairs."
"Skirmish." and "Battle." The only difference I can see between these is the scale of the fight--that is, how many troops are involved, and the duration of the fighting. Offhand, I don't know where the officials drew the line as to how many troops or how short or long the shooting.
"Engagement." I gather this type of fight falls between "Skirmish" and "Battle."
"Action." To me, this is a general term for contact with the enemy that applies to "Skirmish," "Engagement," and "Battle." "Action" is a general term to me.
The terminology for all this has changed in our military in 140 years. What during the Civil War they called "Scouts" we have been calling "Patrols" for many decades. I checked some of my old Field Manuals, but I didn't find a composite modern list for levels of actions with the enemy to parallel this old Civil War list.