There are many response that could be made to that. The one that I personally believe comes down to what the first sin was.
God commanded Adam and Eve not to eat the fruit. But they were ignorant; they didn’t understand good or evil. The serpent entices them to eat the fruit and NOW they know. Now they are able to see good and evil. But they hadn’t sinned yet (James 4:17). God goes for a stroll, calls out for Adam. God asks what happened; and what does Adam say? “The Woman that YOU made me, she did it.” Not my fault, its the woman’s, and oh by the way, your fault too. That was, I believe, the first sin. They shouldn’t have eaten the fruit, but they didn’t know any better. Once they had the knowledge, now they were accountable. And, like we all do, they chose evil over good. That is what God punished.