Allen's conduct and bizarrely tone-deaf statements related to the affair did alienate much of his fan base, a considerable segment of which was female, and women in particular tend to take a poor view of men who mess around with the daughters of the family. There were men who thought he was pretty disgusting, too, of course.
In pictures like Manhattan Allen presented his character as a searcher for enduring values (remember that lecture Michael Murphy got for cheating on his wife with Diane Keaton)? At the end of the movie Allen's search concluded by chasing down his 17-year old lover, a foreshadowing of things to come, perhaps.
The marriage will last as long as Soon-Yi wants it to last, I expect.
Allen has recovered to a considerable extent. Midnight in Paris was his biggest hit. Possibly he's pulling in newer viewers that remember little if anything about the Previn affair, and others have returned to the fold. Laudatory articles and documentaries that politely omit most of the details of the scandal appear regularly. But some fans have long memories. In my view they're not missing much by boycotting him, but reasonable people will disagree.