Overwhelming Zombie attacks

Hello,
so playing skirmish, after an hour or two there are more and more attacks basicly wiping my base soon and ending the game right there. I did not turn on the zombie waves or any other option that would explain that.
Any idea what I missed? Is that "normal" game behaviour?

There is a particular quest (at the Mausoleum?) that has this effect if you or somebody aids the necromancers.

I see, that is maybe possible, there were a lot of quest shrines. Thank you.