This most likely happens because you are trying to show the dialog after the execution of a background thread, while the Activity is being destroyed.
I see this error reported occasionally from some of my apps when the activity calling the dialog completes for some reason or another while trying to display the dialog. This is what solved it for me:
if (!((Activity) context).isFinishing()) { //show dialog }
I've been using this to fix issues on older versions of Android for a few years, and haven't seen a crash since.
2021 Update
It has been noted in some comments that blindly throwing Context to Activity. I agree!
When I write similar code today in Fragment(8+ years after the original answer was given), I do it more like this:
if (!requireActivity().isFinishing) { // show dialog }
The main takeaway is that trying to display a dialog or update any UI after the hosting Activity has been killed will result in a crash. Do what you can to prevent this by killing background threads when your Activity is killed, or at the very least, use the answers here to stop your app from crashing.