Fix: Debug Assertion Failed! When Using CFileDialog Boxes

Symptoms: When using CFileDialog boxes in non-MFC applications, a Debug Assertion Failed! box will appear on-screen, pointing to a file such as afxwin1.inl and offer the typical Abort/Retry/Ignore options. Ignoring the error will allow the application to continue as normal, as though the error didn't mean anything. This same error continues every time you use the functions regardless of the options you change in your code.

I Ran into this situation while working on a File Selector box for one of my DirectX games. I wanted to bring up a file requester to allow a player to save their score files to disk at any location they wanted. After scouring the web, I was not able to find many posts that were helpful, just a lot of people saying I should switch my entire application to MFC, or read the documents properly etc. or that my code was just plain buggy.

The solution was found on a tiny website and didn't give any names other than TheDoc as the author, so I am putting the solution I found to work here so it may help others in the future who wish to use this same type of code. Since applying the fix, I added a whole new series of commands to my personal library that allow me to control file boxes in an easy to use fashion.

The crash basically occurs because the dialog box has no resource to attach to as far as a window instance, MFC apps set all this up behind the scenes and it is never really worried about. Because I am using DirectX, it thinks the handle is bad (which is why the exception is thrown) and resets to default after the alert box finishes, which is why the program will continue as normal after clicking ignore.

To fix the problem, find the WinMain() function of your application and insert this line of code somewhere inside it:

AfxSetResourceHandle(hInstance);

At the top of the same file, you should also add (If not already there):

#ifndef WINVER // Allow use of features specific to Windows XP or later.
#define WINVER 0x0501 // Change this to the appropriate value to target other versions of Windows.
#endif

#include "afxwin.h" // Required for CFileDialog To Work

The WINVER define allows you to compile the various bits and pieces of the CFileDialog for a particular version of Windows. Searching around the MSDN Forums will tell you what values to use if you want something custom. Leaving this define out of the code will set the compiler to Windows Server 2003 (0x0502) as a default.

Recompiling your project and running it should now eliminate the Assertion dialogue box.

If you are using a library that does not give you access to the WinMain() function, I am not sure at this time what can be done to fix the problem. If anyone should have a solution, please post it. Thank-you.