The degree symbol appears fine at first, but then changes to a question mark.

This is a bug in Prism 4 Mac. We'll fix it in release 4.0c. As a workaround, don't use the special degree character that is on the Math symbols that Prism offers. Instead offer a lower case 'o' and superscript it.