I would say that your position is very close to that of the left communists: we are against fascism, obviously, but we reject the notion of the 'lesser evil', which means allying with liberal factions of the bourgeoisie against fascism. Fascism has to to be opposed by class struggle - as do the 'liberal' factions. Here is a classic text from the Italian left communists of the 1930s, with an intro by the ICC:
http://en.internationalism.org/ir/101_bilan.htm