As I remember when Major Robert Anderson abandoned Fort Moultire and occupied Fort Sumter, I believe that his first justification was that he felt that Fort Moultrie was indefencible and was afraid that the South Carolina militia and citizens of Charleston would attack him there.
As I look at a map of Charleston Harbor, Fort Moultrie was located on Sullivan Island which is surrounded by the waters of Charleston Harbor, the Atlantic Ocean, and a swamp. How were the people of Charleston going to attack Fort Moultrie without the use of boats? The same way they would have had to attack Fort Sumter. How much of a real threat could the people of Charleston have been?