Wayne Ahmad was serving a community order for a similar attack when he let himself into their house and touched them.
Jailing him for four years, Judge Tom Bayliss QC said the mother and daughter were entitled to be safe in their own home.
He said: “The impact on your victims was profound.”
Leeds Crown Court heard last Friday that Ahmad, 41, had been at a barbecue at their house.
The woman told him to leave her alone when he said he fancied her and touched her breast.
Ahmad continued drinking elsewhere after he left the house and returned later that night.
The woman was asleep beside her husband when she woke to find Ahmad touching her. His phone began to ring at that point and the woman believed he had gone out the house when he then left the room.
But Ahmad, of Greenside Estate, Mirfield, went into the bedroom of the woman’s 13-year-old daughter.
She woke to find him touching her and told him to stop.
He moved away and appeared to be using his phone, but the girl realised he was touching himself.
Ahmad was found guilty of three counts of sexual assault and one of engaging in sexual activity in the presence of a child after a trial.
Martin Sharpe, mitigating, said Ahmad had an “extremely difficult childhood” and his drinking was a concern.
He said his client had not committed any further offences since the assault on the mother and daughter in April 2009.
The court heard Ahmad had been subject to an 18 month community order for a similar sexual assault at the time of the offences. In that case he had gained access to the home of a woman he was working for as a builder and assaulted her as she slept.
Judge Bayliss said: “There are worrying aspects about you and your behaviour.”
But he said that an extended sentence would not be appropriate because there had been no more offences since 2009.
Ahmad was given an indefinite sexual offences prevention order and told to sign the sex offenders’ register for life.