There are a lot of sinks and such in that area. One, Devil's Millhopper near Gainesville, is quite beautiful. I've never heard of them being used to confine prisoners, but I suppose it is possible. Most prisoners captured were moved on to Andersonville pretty fast.
The Confederacy did explore some Florida caves to evaluate their potential for use in mining raw material for gunpowder. The efforts found most caves in the state too wet to be of much use. I know that they investigated caves near Marianna and Gainesville during this project.
Best,
Dale Cox