The same thing is happening to me, I had talked about it here:
I gave up on moving my young Zoi to that house.
After a few days I noticed that a new family (one of those created by the game) moved into that house. I checked and the babies are still there on the floor.
As you can see in my case, even deleting the house didn’t work.