I found a workaround for my project, although it's not universally usable.
In core/components/quip/model/quip/quipcomment.class.php
around line 108, change this line:
$options['context_key'] = $this->get('context_key');
to
$options['context_key'] = 'web';
I figured that it must be looking in the wrong context, because no context-setting is found. And if there's no setting, the current context is used. In my case the wrong context. So I changed this line to force it to look into the web context.
How can I set $options['context_key']?