States have coats of arms and state seals (they are sometimes different) and those states in ante-bellum America have very old COAs. For example, Georgia's dates to 1799 and is still in use today.
It was not uncommon to have such devices on flags and drums, etc. The flags of the Virginia militia that pre-date the Civil War have the state COA in the center as did the militia flags of South Carolina (dating to 1839 with their Militia Act). Northern states also did the same at times.