Pharaoh "not knowing" Joseph is where the trouble starts, leading very quickly to an attempt to exterminate the People.