Fairness in government spending 'should not be judged simply by size of handouts'