The man broke into a property on Ravensthorpe Road, Thornhill Lees, at around 3.45am on Wednesday, August 12.
A 36-year-old woman was woken up by the noise and met the suspect on the upstairs hallway.
He told her he had the wrong address and left through the front door without stealing anything.
Detectives have now released an e-fit image of a man they would like to speak to in connection with the incident.
He is described as Asian, aged around 20-years-old, short and of medium build.
Det Insp Andy Leonard, of Kirklees CID, said: “We would urge anyone who recognises the man pictured in this e-fit image to please get in touch.
“While none of the occupants of the house were harmed during this incident, it was clearly an extremely frightening experience for them and it is vital that we catch the man responsible.
“Anyone with information which could help the ongoing investigation is asked to call Kirklees CID via 101 or Crimestoppers, anonymously, on 0800 555 111.”