"I have often wondered why most of the mounted soldiers did not clear out"

You are correct we may have a bit of an explaination here in Porters account. There is very little use for mounted Cavalry in siege warfare. And Porter comments that the horses were inside the Fort. It was probably thought to be the safest place to keep them.