The principle of explosion ought to be rejected. Given how many beliefs we all have, it is very likely true that at least two of them together form a contradiction (either directly or inferentially) - it is unlikely that the set of beliefs that any of us in fact holds is totally consistent - we do try to be as consistent as we can be, but it is impossible for mere humans to be perfectly consistent, for there often exist contradictions which are not immediately apparent. So, the conjunction of all our beliefs very likely constitutes or implies a contradiction. By the principle of explosion, from such a contradiction, any belief logically follows. Hence, it is very likely that we are logically justified in believing anything whatsoever. Clearly, this is absurd - where did we go wrong? I think we went wrong in accepting the principle of explosion.
The purpose of logic should be to model human thought, and improve it through formalisation and regularisation. But if the model, the formalisation, yields conclusions that contradict common sense, then should we reject common sense, or should we go looking for an alternative model which corresponds with it better?
Classical logic is based on material implication, for which it is true that “from a falsehood anything follows” (ex falso quodlibet). The principle of explosion, “from a contradiction anything follows” (ex contradictione quodlibet), is just a special case of this more general principle of material implication. As an example of an application of that principle, consider the following inference: “If John F. Kennedy was President of the Soviet Union, then dinosaurs live on the moon”. Now, the antecedent is false, therefore according to material implication, the conditional is true, no matter how absurd the consequent - indeed, despite the fact that the consequent has nothing to do with the antecedent, and that even were the antecedent true, that would give us no reason to believe the consequent. So, if material implication violates common sense, which should yield: common sense, or material implication? I say, keep common sense, and get rid of material implication. But since classical logic is based on material implication, that means we need to abandon classical logic and turn to non-classical logics such as relevance logics (which are a type of paraconsistent logic). And in relevance logics and paraconsistent logics, the principle of explosion is rejected.