Put simply, Batman doesn't commit murder because he refuses to intentionally take a life with his own hands and become an executioner. The basic answer is easy enough to articulate. But the reason behind it is very complicated.
Bruce Wayne witnessed the simple power of taking a life, when he was a child watching his parents die in front of him. The act itself is easy, something anyone can really do if they want to, but the impact of murder is complex and monumental, because the implications of an execution last forever.
Joe Chill shot Bruce's parents in a moment of fear and desperation, just to grab some money and without the intention of taking anyone's life -- but his simple act of reflexively pulling a trigger, in a split second, forever changed the world through the ripples it sent out, taking the Wayne's from the world, ending Thomas Wayne's medical practice, ending the parents' philanthropy, and of course sending Bruce on a path inescapably toward becoming Batman.
Bruce is aware of this with every fiber of his being. He relives that murder in his darkest moments, and it is in the memory and honor of his parents that he fights to make their city a better place. He can never become the thing that struck them down, a murderer who takes the simple path that sends out those endless ripples. The purposeful taking of another human life, to assume the power and responsibility of forever ending a life, is the defining event against which Batman rose to resist. The moment he takes a life, he has lost his reason to exist, because he will have become the very thing he was born to end.
Bruce accepts that he must be a criminal, a vigilante, to do his work. But this he accepts as an unavoidable element of his mission -- to stand against the peculiar and very special circumstances of Gotham's corruption that reaches to the highest government offices and taints the justice system and law enforcement, Batman would necessarily have to operate outside of the legal system. To be free of outside influences and accountable only to himself and his mission, too, required being an outlaw of sort.
But this lack of accountability is also a burden, and it means he must carefully weigh his actions and police himself as much as he polices the city -- and he is well aware that without any other accountability or authority, he must restrict himself from actions that go too far and that are outside the ultimate goal of standing for the ideal of the rule of law as a social contract. A man accountable only to himself might be able to excuse actions that can be corrected or amended if he is wrong, but when he is answerable only to himself, the temptation to weigh the lives of others and to deal out judgment as a God deciding the ultimate fate of victims, is too much. A line must be drawn, or there will eventually be no line at all and no limitation on his own actions, because he will stop being accountable even to himself if he does not hold himself accountable for the irreversible and absolute exercise of authority over life and death.
It is easy -- too easy -- to think that yes, the Joker has killed so many victims and escaped so many times, the only way to make him stop killing innocent civilians is to just kill him. In Frank Miller's The Dark Knight Returns, there is a great moment when, after the Joker has detonated a hidden bomb in an apartment complex, Batman thinks to himself that he will stay and help the police pull people out of the rubble and do the best he can, and then he'll count the dead and add them to the list of all of the people he himself has murdered by letting the Joker live. So Batman doesn't fail to understand the horrible math involved, the terrible moral trap that presents itself the moment he begins to let himself even consider the possibility of killing the Joker in order to save future lives.
[Let me take a moment to comment on that last point, just to be clear: it is ONLY in the context of preventing future presumed murders, of course, that we can even begin to talk about the moral dilemma of "why doesn't Batman kill the Joker," because the notion of killing the Joker out of revenge or to make him pay for his past crimes doesn't enter into the equation. Punishment isn't what Batman is about, it's not his mission, it's not his mental frame of reference for what he does and why he does it. And he does on some level realize that the Joker's madness and psychosis are so absolute and pure that "punishment" itself is an irrational response to someone who exists beyond such concepts like the Joker. It is only the idea of killing the Joker to preserve other lives in the future, then, that we are talking about here.]
While Batman realizes the implications of the mathematical calculations, that refusing to execute the Joker will almost surely mean more innocent deaths if the Joker escapes again (and he always does, eventually), he also realizes that if the point is that the Joker's life isn't as valuable as the lives of his victims -- so much so that the mere CHANCE of more victims is enough to justify murdering the Joker -- then a single innocent life should be enough to justify murdering the Joker. It cannot be a case of weighing the number of innocent lives, if innocent life is so precious it justifies murder then even one should be too many, since the equation is one life (a victim's) versus one life (the Joker's).
And if one innocent life is too valuable, and would justify taking the life of a killer, then the same equation applies to most of Batman's other arch villains as well. And it applies to the mobsters. And it applies to any killers. And, if the equation is one in which the likelihood and chance of future victims is enough to justify murdering someone to stop the potential/likelihood of future victims, then rationally chronic drunk drivers and armed robbers and many others also qualify.
Which is where the lack of a prohibition means the lack of a clear line, which means the lack of ANY line, which means the lack of any accountability other than to himself necessarily leads -- irresistibly and unstoppably -- to lack of accountability even to himself. He becomes absolute, and murder of anyone becomes justifiable. Because once you've justified the above examples, you place in your own hands a presumption of moral certainty (which you MUST presume, you MUST feel with absolute certainty, or you can never trust yourself to begin murdering based on who you believe should be murdered) that will eventually lead to the same certainty about your best guess, about your gut feeling, and the basic minimum standard becomes so arbitrary that there IS no minimum standard anymore. Once the idea enters your head that a person should die, your absolute moral authority, without accountability whatsoever, means you can find a reason to kill them if you wish to do so.
That is the logical progression, in the context of one man placing himself beyond all outside accountability and authority, and granting himself the latitude to judge when it is acceptable to murder other people. The mathematics is a ruse, a distraction to tempt and allow that first taste of blood, that first deceptively easy step across a line that vanishes forever once you cross it. If the loss of innocent life is enough reason to execute another life judged not innocent, then one innocent life becomes enough reason to execute another life judged less innocent.
Batman knows this, because he lives it every day -- he lives it by watching it transpire in Gotham, by fighting it when it manifests daily in the villains and in the hearts of ordinary citizens and in the minds of cops with a badge and a gun and a creeping sense that it's so easy to justify taking a life to stop future wrongdoing. But most of all, he lives it because it resides in his own mind, every single day, when he must remember that terrible moment as a child when he watched another man make the calculation of Bruce's parents' lives versus a few dollar bills. And he lives it because within his own heart, he hears that little whisper tempting him to cross the line, with the Joker or with the Penguin or with the mobsters or with a serial killer.
So he refuses to become a murderer, because he knows that murdering the Joker leads to murdering all of them, making each killing easier than the last. And that casts him as the very thing that created him, as the thing he fights against, because at that point the only difference between Batman and the Joker would be that Batman thinks he's able to justify his own murders.
More questions on DC Comics: