Have the local UDC members, especially relatives of those buried in the cemetary, show up to visit their ancestors graves or the monument on Memorial Day. Put Confederate flags on every Confederate's grave. Memorial day is on a Monday so it should not interfere with any church service. If there are any Union graves or U.S. service men buried in the cemetary, put an American flag on their plot.
Have someone stay all day with a camera and take pictures of anyone removing the flags.

Start a ruckus.

You have every right to honor soldiers on Memorial Day.