To better explain the situation, it's important to understand that in the autumn of 1864, Georgia farmers were harvesting a larger than normal crop. Governor Brown had encouraged citizens to plant more than usual in order to help feed the Confederate Army of Tennessee, then camped in north Georgia. Sherman's men got most of the benefit of this crop as they marched east from Atlanta in three columns. Most reports suggest that in addition to crops being harvested, livestock and poultry were plentiful.
However, most of the harvest had not been collected at commissary depots. Food supplies, although ample, still had to be gathered from individual farms. If Sherman's armies camped in any one place more than a few days, all crops and livestock within reach would had been consumed, and soldiers would face starvation. Retreat wasn't an option, because everything in their path had been consumed or destroyed. To keep each of his columns fed, Sherman had to keep them moving towards Savannah.
Farms in Sherman's path had enough to feed his men and animals, but not the slaves who attempted to follow his armies.
To return to Federal POWs and their guards at Andersonville, Millen GA and Florence SC, your remarks about their provisions were correct. Georgia senior reserves assigned to duty as prison guards received the same rations as prisoners, and that wasn't much. Given the large harvest available on nearby Georgia farms, why couldn't they have eaten better?
The answer has to do with a point raised earlier, that the harvest had yet to be collected at depots for distribution. In addition, the Confederate transportation system, subject to frequent interruptions and failures, could never meet the needs of the army or Federal prisoners held at Andersonville, Millen and Florence. Federal cavalry raids during late July and August made matters worse.
Poor sanitary conditions caused men in camps to suffer from dysentery &c., further compounding the problem of malnutrition. Confederate soldiers sent to overworked Georgia hospitals reported many of the same health problems.
As far as the Confederate government's official position regarding prisoners of war, this may be worth noting. During 1864 the War Department in Richmond sent three different inspectors to report on conditions and operation of the camp at Andersonville. Inspection reports such as these were intended to provide detailed information for the War Department. They were not meant to be available to Congress or other public forums. Accordingly, inspectors were encouraged to offer opinions based on interviews of officers and observations made during their inspections. These are available in the OR series or the NARA microfilm set of Confederate inspection reports.