There's something goofy going on with the caching. I changed it to:
[[!~[[getContextSetting? &context=`someContext` &setting=`site_start`]]]]
and it worked for all contexts. Then I removed the ! and it still worked. I can't explain why.
BTW, it's a good idea to add ? &scheme=`full` like this:
[[!~[[getContextSetting? &context=`someContext` &setting=`site_start`]]? &scheme=`full`]]
Be sure you've actually created the Context Setting for all contexts. The snippet will never return the site_start System Setting (unless you alter it to do so).