You are paying for data stored. Which is basically inputs, outputs, and headers. 

Please have a look on the schema above. 
Output B (of original TX) is the same as input B (of CPFP TX). When you remove them you save on fees. Both are redundant.
And you don't need even the second TX header since it will be one transaction now. 

In that case don't both the original TX fees and the CPFP TX both need to be paid? How would that save fees for A?

Yes. It's CPFP - ancestors and descendant merged into one transaction... And all your pending transactions too..