There are two interconnected reasons, I think.
1) The founders of this quasi-religion weren't workers; they came from a class that educated their children in the classics and in philosophy. This tradition taught that behind appearances there is a hidden world, accessible to thought alone, which is more real than the material universe we see around us.
This way of seeing things was invented by ideologues of the ruling class, who viewed reality this way. They invented it because if you belong to, benefit from or help run a society which is based on gross inequality, oppression and exploitation, you can keep order in several ways.
The first and most obvious way is through violence. This will work for a time, but it is not only fraught with danger, it is costly and it stifles innovation (among other things).
Another way is to persuade the majority (or a significant section of "opinion formers" and administrators, at least) that the present order either works for their benefit, is ordained of the 'gods', or that it is 'natural' and cannot be fought, reformed or negotiated with.
Hence, a world-view is necessary for the ruling-class to carry on ruling in the same old way. While the content of this ruling ideology may have changed with each change in the mode of production, its form has remained largely the same for thousands of years: Ultimate Truth is ascertainable by thought alone, and it can therefore be imposed on reality dogmatically
So, these non-worker founders of our movement, who had been educated to believe there was just such a hidden world that governed everything, looked for principles in that invisible world that told them that change was inevitable, and part of the cosmic order. Enter dialectics, courtesy of the dogmatic ideas of a ruling-class mystic called Hegel.
2) That allowed the founders of this quasi-religion to think of themselves as special, as prophets of the new order, which workers, alas, could not quite grasp because of their defective education and their reliance on ordinary language and 'common sense'.
Fortunately, history has predisposed these prophets to ascertain the truth about reality for the rest of us, which means that they are our 'naturally-ordained' leaders. That in turn meant these 'leaders' were also teachers of the 'ignorant masses' and who could thus legitimately substitute themselves for the unwashed majority -- in 'their own interests'
, you understand. This is because the masses are too caught up in 'commodity fetishism' to see the truth for themselves.
And that is why DM is a world-view.
It is also why dialecticians cling on to this theory like grim death (and become very emotional (and abusive!) when it is attacked by yours truly), since it provides them with a source of consolation that, despite outward appearances to the contrary, and because this hidden world tells them that Dialectical Marxism will one day be a success, everything is in fact peachy, and nothing in the core theory needs changing -- in spite of the fact that that core theory says everything changes! Hence, it is ossified into a dogma, and imposed on reality. A rather nice unity of opposites for you to ponder.
So, this 'theory' insulates the militant mind from the facts
; it tells such comrades that reality 'contradicts' outward appearances. Hence, even if Dialectical Marxism appears
to be a long-term failure, those with a the equivalent of a dialectical 'third eye' can see the opposite is in fact the case: Dialectical Marxism is a ringing success!
In that case, awkward facts can either be ignored or they can be re-configured into their opposite.
In that case:
Dialectics is the sigh of the depressed dialectician, the heart of a heartless world. It is the opiate of the party. The abolition of dialectics as the illusory happiness of the party hack is required for their real happiness. The demand to give up the illusion about its condition is the demand to give up a condition which needs illusions.
Unfortunately, these sad characters will need (materialist) workers to rescue them from themselves.
I stand no chance...