The PM has had a bruising week that has seen him suffer damaging parliamentary defeats and the resignation of his own brother.