Analysis:

The example will display a dialog box with three edit controls that will be filled with the class name, window handle and the address of the window procedure associated with the window under the mouse cursor. There are two buttons, Hook and Exit. When you press the Hook button, the program hooks the mouse input and the text on the button changes to Unhook. When you move the mouse cursor over a window, the info about that window will be displayed in the main window of the example. When you press Unhook button, the program removes the mouse hook.

The main program uses a dialog box as its main window. It defines a custom message, WM_MOUSEHOOK which will be used between the main program and the hook DLL. When the main program receives this message, wParam contains the handle of the window that the mouse cursor is on. Of course, this is an arbitrary arrangement. I decide to send the handle in wParam for the sake of simplicity. You can choose your own method of communication between the main program and the hook DLL. .IF HookFlag==FALSE invoke InstallHook,hDlg .IF eax!=NULL mov HookFlag,TRUE invoke SetDlgItemText,hDlg,IDC_HOOK,ADDR UnhookText .ENDIF

The program maintains a flag, HookFlag, to monitor the state of the hook. It's FALSE if the hook is not installed and TRUE if the hook is installed.

When the user presses Hook button, the program checks if the hook is already installed. If it is not, it call InstallHook function in the hook DLL to install it. Note that we pass the handle of the main dialog as the parameter of the function so the hook DLL can send the WM_MOUSEHOOK messages to the right window i.e. our own.
When the program is loaded, the hook DLL is loaded too. Actually, DLLs are loaded immediately after the program is in memory. The DLL entrypoint function is called before the first instruction in the main program is execute even. So when the main program executes the DLL(s) is/are initialized. We put the following code in the DLL entrypoint function of the hook DLL:

The code just saves the instance handle of the hook DLL itself to a global variable named hInstance for use within the InstallHook function. Since the DLL entrypoint function is called before other functions in the DLL are called, hInstance is always valid. We put hInstance in .data section so that this value is kept on per-process basis. Since when the mouse cursor hovers over a window, the hook DLL is mapped into the process. Imagine that there is already a DLL that occupies the intended load address of the hook DLL, the hook DLL would be remapped to another address. The value of hInstance will be updated to those of the new load address. When the user presses Unhook button and then Hook button, SetWindowsHookEx will be called again. However, this time, it will use the new load address as the instance handle which will be wrong because in the example process, the hook DLL's load address hasn't been changed. The hook will be a local one where you can hook only the mouse events that occur in your own window. Hardly desirable.

The InstallHook function itself is very simple. It saves the window handle passed as its parameter to a global variable named hWnd for future use. It then calls SetWindowsHookEx to install a mouse hook. The return value of SetWindowsHookEx is stored in a global variable named hHook for use with UnhookWindowsHookEx.
After SetWindowsHookEx is called, the mouse hook is functional. Whenever a mouse event occurs in the system, MouseProc (your hook procedure) is called.

The first thing it does is to call CallNextHookEx to give the other hooks the chance to process the mouse event. After that, it calls WindowFromPoint function to retrieve the handle of the window at the specified screen coordinate. Note that we use the POINT structure in the MOUSEHOOKSTRUCT structure pointed to by lParam as the current mouse coordinate. After that we send the window handle to the main program via PostMessage with WM_MOUSEHOOK message. One thing you should remember is that: you should not use SendMessage inside the hook procedure, it can cause message deadlock. PostMessage is recommended. The MOUSEHOOKSTRUCT structure is defined below:

• pt is the current screen coordinate of the mouse cursor.
• hwnd is the handle of the window that will receive the mouse message. It's usually the window under the mouse cursor but not always. If a window calls SetCapture, the mouse input will be redirected to that window instead. Because of this reason, I don't use the hwnd member of this structure but choose to call WindowFromPoint instead.
• wHitTestCode specifies the hit-test value. The hit-test value gives more information about the current mouse cursor position. It specifies on what part of window the mouse cursor is. For complete list, check your win32 api reference under WM_NCHITTEST message.
• dwExtraInfo contains the extra information associated with the message. Normally this value is set by calling mouse_event and retrieved by calling GetMessageExtraInfo.

To avoid flickers, we check the text already in the edit controls and the text we will put into them if they are identical. If they are, we skip them.
We retrieve the class name by calling GetClassName, the address of the window procedure by calling GetClassLong with GCL_WNDPROC and then format them into strings and put them into the appropriate edit controls.

When the user presses Unhook button, the program calls UninstallHook function in the hook DLL. UninstallHook just calls UnhookWindowsHookEx. After that, it changes the text of the button back to "Hook", HookFlag to FALSE and clears the content of the edit controls.
Note the linker switch in the makefile.

It specifies .bss section as a shared section to make all processes share the same uninitialized data section of the hook DLL. Without this switch, your hook DLL will not function correctly.