Iczelion's Win32 Assembly Tutorial 21: Pipe

Tutorial 21: Pipe

In this tutorial, we will explore pipe, what it is and what we can use it for. To make it more interesting, I throw in the technique on how to change the background and text color of an edit control.

Download the example here.

Theory:

Pipe is a communication conduit or pathway with two ends. You can use pipe to exchange the data between two different processes, or within the same process. It's like a walkie-talkie. You give the other party one set and he can use it to communicate with you.

There are two types of pipes: anonymous and named pipes. Anonymous pipe is, well, anonymous: that is, you can use it without knowing its name. A named pipe is the opposite: you have to know its name before you can use it.

You can also categorize pipes according to its property: one-way or two-way. In a one-way pipe, the data can flow only in one direction: from one END to the other. While in a two-way pipe, the data can be exchanged between both ends.

An anonymous pipe is always one-way while a named pipe can be one-way or two-way. A named pipe is usually used in a network environment where a server can connect to several clients.

In this tutorial, we will examine anonymous pipe in some detail. Anonymous pipe's main purpose is to be used as a communcation pathway between a parent and child processes or between child processes.

Anonymous pipe is really useful when you deal with a console application. A console application is a kind of win32 program which uses a console for its input & output. A console is like a DOS box. However, a console application is a fully 32-bit program. It can use any GUI function, the same as other GUI programs. It just happens to have a console for its use.

A console application has three handles it can use for its input & output. They are called standard handles. There are three of them: standard input, standard output and standard error. Standard input handle is used to read/retrieve the information from the console and standard output handle is used to output/print the information to the console. Standard error handle is used to report error condition since its output cannot be redirected.

A console application can retrieve those three standard handles by calling GetStdHandle function, specifying the handle it wants to obtain. A GUI application doesn't have a console. If you call GetStdHandle, it will return error. If you really want to use a console, you can call AllocConsole to allocate a new console. However, don't forget to call FreeConsole when you're done with the console.

Anonymous pipe is most frequently used to redirect input and/or output of a child console application. The parent process may be a console or a GUI application but the child must be a console app. for this to work. As you know, a console application uses standard handles for its input and output. If we want to redirect the input and/or output of a console application, we can replace the handle with a handle to one END of a pipe. A console application will not know that it's using a handle to one END of a pipe. It'll use it as a standard handle. This is a kind of polymorphism, in OOP jargon. This approach is powerful since we need not modify the child process in anyway.

Another thing you should know about a console application is where it gets those standard handles from. When a console application is created, the parent process has two choices: it can create a new console for the child or it can let the child inherit its own console. For the second approach to work, the parent process must be a console application or if it's a GUI application, it must call AllocConsole first to allocate a console.

Let's begin the work. In order to create an anonymous pipe you need to call createpipe.

CreatePipe has the following prototype:

CreatePipe PROTO pReadHandle:DWORD,    \
                 pWriteHandle:DWORD,   \
                 pPipeAttributes:DWORD,\
                 nBufferSize:DWORD

preadhandle is a pointer to a dword variable that will receive the handle to the read END of the pipe.
pwritehandle is a pointer to a dword variable that will receive the handle to the write END of the pipe.
ppipeattributes points to a SECURITY_ATTRIBUTES structure that determines whether the returned read & write handles are inheritable by child processes.
nbuffersize is the suggested size of the buffer the pipe will reserve for use. This is a suggested size only. You can use NULL to tell the function to use the default size.

If the call is successful, the return value is nonzero. If it failed, the return value is zero.
After the call is successful, you will get two handles, one to read END of the pipe and the other to the write END. Now I will outline the steps needed for redirecting the standard output of a child console program to your own process. Note that my method differs from the one in Borland's win32 api reference. The method in win32 api reference assumes the parent process is a console application and thus the child can inherit the standard handles from it. But most of the time, we will need to redirect output from a console application to a GUI one.

Create an anonymous pipe with CreatePipe. Don't forget to set the bInheritable member of SECURITY_ATTRIBUTES to TRUE so the handles are inheritable.
Now we must prepare the parameters we will pass to CreateProcess since we will use it to load the child console application. One important structure is the STARTUPINFO structure. This structure determines the appearance of the main window of the child process when it first appears. This structure is vital to our purpose. You can hide the main window and pass the pipe handle to the child console process with it. Below is the members you must fill:
- cb : the size of STARTUPINFO structure
- dwFlags : the binary bit flags that determine which members of the structure are valid also it governs the show/hide state of the main window. For our purpose, you should use a combination of STARTF_USESHOWWINDOW and STARTF_USESTDHANDLES
- hStdOutput and
  hStdError : the handles you want the child process to use as standard output/error handles. For our purpose, we will pass write handle of the pipe as the standard output and error of the child. So when the child outputs something to the standard output/error, it actually passes the info via the pipe to the parent process.
- wShowWindow governs the show/hide state of the main window. For our purpose, we don't want the console window of the child to show so we put SW_HIDE into this member.
Call CreateProcess to load the child application. After CreateProcess is successful, the child is still dormant. It is loaded into memory but it doesn't run immediately
Close the write pipe handle. This is necessary. Because the parent process has no use for the write pipe handle, and the pipe won't work if there are more than one write END, we MUST close it before reading the data from the pipe. However, don't close the write handle before calling CreateProcess, your pipe will be broken. You should close it just after CreateProcess returns and before you read data from the read END of the pipe.
Now you can read data from the read END of the pipe with ReadFile. With ReadFile, you kick the child process into running mode. It will start execution and when it writes something to the standard output handle (which is actually the handle to the write END of the pipe), the data are sent through the pipe to the read END. You can think of ReadFile as sucking data from the read END of the pipe. You must call ReadFile repeatedly until it returns 0 which means there are no more data to be read. You can do anything with the data you read from the pipe. In our example, I put them into an edit control.
Close the read pipe handle.

Example:

.386
.model FLAT,STDCALL
OPTION casemap:none

include    \masm32\include\windows.inc
include    \masm32\include\user32.inc
include    \masm32\include\kernel32.inc
include    \masm32\include\gdi32.inc
includelib \masm32\lib\gdi32.lib
includelib \masm32\lib\user32.lib
includelib \masm32\lib\kernel32.lib

WinMain PROTO :DWORD,:DWORD,:DWORD,:DWORD

.const
IDR_MAINMENU equ 101         ; the ID of the main menu
IDM_ASSEMBLE equ 40001

.data
ClassName          db "PipeWinClass",0
AppName            db "One-way Pipe Example",0 EditClass db "EDIT",0
CreatePipeError    db "Error during pipe creation",0
CreateProcessError db "Error during process creation",0
CommandLine        db "ml /c /coff /Cp test.asm",0

.data?
hInstance HINSTANCE ?
hwndEdit  dd        ?

.code
start:
        invoke GetModuleHandle, NULL
        mov    hInstance,eax
        invoke WinMain, hInstance,NULL,NULL, SW_SHOWDEFAULT
        invoke ExitProcess,eax

WinMain PROC hInst:DWORD,hPrevInst:DWORD,CmdLine:DWORD,CmdShow:DWORD
LOCAL wc:WNDCLASSEX
LOCAL msg:MSG
LOCAL hwnd:HWND
        mov    wc.cbSize,SIZEOF WNDCLASSEX
        mov    wc.style, CS_HREDRAW or CS_VREDRAW
        mov    wc.lpfnWndProc, OFFSET WndProc
        mov    wc.cbClsExtra,NULL
        mov    wc.cbWndExtra,NULL
        push   hInst
        pop    wc.hInstance
        mov    wc.hbrBackground,COLOR_APPWORKSPACE
        mov    wc.lpszMenuName,IDR_MAINMENU
        mov    wc.lpszClassName,OFFSET ClassName
        invoke LoadIcon,NULL,IDI_APPLICATION
        mov    wc.hIcon,eax
        mov    wc.hIconSm,eax
        invoke LoadCursor,NULL,IDC_ARROW
        mov    wc.hCursor,eax
        invoke RegisterClassEx, ADDR wc
        invoke CreateWindowEx,WS_EX_CLIENTEDGE,ADDR ClassName, \
                                               ADDR AppName,   \
                              WS_OVERLAPPEDWINDOW+WS_VISIBLE,  \
                              CW_USEDEFAULT,CW_USEDEFAULT,     \
                              400,200,NULL,NULL,hInst,NULL
        mov    hwnd,eax
.WHILE TRUE
        invoke GetMessage, ADDR msg,NULL,0,0
.BREAK .IF (!eax)
        invoke TranslateMessage, ADDR msg
        invoke DispatchMessage, ADDR msg
.ENDW
        mov eax,msg.wParam
        ret
WinMain ENDP

WndProc PROC hWnd:HWND, uMsg:UINT, wParam:WPARAM, lParam:LPARAM
LOCAL rect:RECT
LOCAL hRead:DWORD
LOCAL hWrite:DWORD
LOCAL startupinfo:STARTUPINFO
LOCAL pinfo:PROCESS_INFORMATION
LOCAL buffer[1024]:byte
LOCAL bytesRead:DWORD
LOCAL hdc:DWORD
LOCAL sat:SECURITY_ATTRIBUTES
.IF uMsg==WM_CREATE
        invoke CreateWindowEx,NULL,ADDR EditClass, NULL,        \
                              WS_CHILD+ WS_VISIBLE+ES_MULTILINE \
                            + ES_AUTOHSCROLL+ ES_AUTOVSCROLL,   \
                              0,0,0,0,                          \
                              hWnd, NULL, hInstance,NULL
        mov    hwndEdit,eax
.ELSEIF uMsg==WM_CTLCOLOREDIT
        invoke SetTextColor,wParam,Yellow
        invoke SetBkColor,wParam,Black
        invoke GetStockObject,BLACK_BRUSH
        ret
.ELSEIF uMsg==WM_SIZE
        mov    edx,lParam
        mov    ecx,edx
        shr    ecx,16
        and    edx,0ffffh
        invoke MoveWindow,hwndEdit,0,0,edx,ecx,TRUE
.ELSEIF uMsg==WM_COMMAND
.IF lParam==0
        mov    eax,wParam
.IF ax==IDM_ASSEMBLE
        mov    sat.nLength,sizeof SECURITY_ATTRIBUTES
        mov    sat.lpSecurityDescriptor,NULL
        mov    sat.bInheritHandle,TRUE
        invoke CreatePipe,ADDR hRead,ADDR hWrite,ADDR sat,NULL
.IF eax==NULL
        invoke MessageBox, hWnd, ADDR CreatePipeError, \
                                 ADDR AppName, MB_ICONERROR+MB_OK
.ELSE
        mov    startupinfo.cb,sizeof STARTUPINFO
        invoke GetStartupInfo,ADDR startupinfo
        mov    eax, hWrite
        mov    startupinfo.hStdOutput,eax
        mov    startupinfo.hStdError,eax
        mov    startupinfo.dwFlags, STARTF_USESHOWWINDOW \
                                   +STARTF_USESTDHANDLES
        mov    startupinfo.wShowWindow,SW_HIDE
        invoke CreateProcess, NULL, ADDR CommandLine,    \
                              NULL, NULL, TRUE, NULL, NULL,
                              NULL, ADDR startupinfo, ADDR pinfo
.IF eax==NULL
        invoke MessageBox,hWnd,ADDR CreateProcessError,ADDR AppName, \
                          MB_ICONERROR+MB_OK
.ELSE
        invoke CloseHandle,hWrite
.WHILE TRUE
        invoke RtlZeroMemory,ADDR buffer,1024
        invoke ReadFile,hRead,ADDR buffer,1023,ADDR bytesRead,NULL
.IF eax==NULL
.BREAK
.ENDIF
        invoke SendMessage,hwndEdit,EM_SETSEL,-1,0
        invoke SendMessage,hwndEdit,EM_REPLACESEL,FALSE,ADDR buffer
.ENDW
.ENDIF
        invoke CloseHandle,hRead
.ENDIF
.ENDIF
.ENDIF
.ELSEIF uMsg==WM_DESTROY
        invoke PostQuitMessage,NULL
.ELSE
        invoke DefWindowProc,hWnd,uMsg,wParam,lParam ret
.ENDIF
        xor    eax,eax
        ret
WndProc ENDP

END start

Analysis:

The example will call ml.exe to assemble a file named test.asm and redirect the output of ml.exe to the edit control in its client area.
When the program is loaded, it registers the window class and creates the main window as usual. The first thing it does during main window creation is to create an edit control which will be used to display the output of ml.exe.
Now the interesting part, we will change the text and background color of the edit control. When an edit control is going to paint its client area, it sends WM_CTLCOLOREDIT message to its parent.
wParam contains the handle to the device context that the edit control will use to write its own client area. We can use this opportunity to modify the characteristics of the HDC. .ELSEIF uMsg==WM_CTLCOLOREDIT invoke SetTextColor,wParam,Yellow invoke SetTextColor,wParam,Black invoke GetStockObject,BLACK_BRUSH ret SetTextColor changes the text color to yellow. SetTextColor changes the background color of the text to black. And lastly, we obtain the handle to the black brush which we return to Windows. With WM_CTLCOLOREDIT message, you must return a handle to a brush which Windows will use to paint the background of the edit control. In our example, I want the background to be black so I return the handle to the black brush to Windows.

Assemble menuitem, it creates an anonymous pipe.

.IF ax==IDM_ASSEMBLE
        mov    sat.nLength,sizeof SECURITY_ATTRIBUTES
        mov    sat.lpSecurityDescriptor,NULL
        mov    sat.bInheritHandle,TRUE

Prior to calling CreatePipe, we must fill the SECURITY_ATTRIBUTES structure first. Note that we can use NULL in lpSecurityDescriptor member if we don't care about security. And the bInheritHandle member must be TRUE so that the pipe handles are inheritable to the child process. invoke CreatePipe,ADDR hRead,ADDR hWrite,ADDR sat,NULL ; After that, we call CreatePipe which, if successful, will fill ; hRead and hWrite variables with the handles to read and write ends ; of the pipe respectively. mov startupinfo.cb,sizeof STARTUPINFO invoke GetStartupInfo,ADDR startupinfo mov eax, hWrite mov startupinfo.hStdOutput,eax mov startupinfo.hStdError,eax mov startupinfo.dwFlags, STARTF_USESHOWWINDOW \ +STARTF_USESTDHANDLES mov startupinfo.wShowWindow,SW_HIDE Next we must fill the STARTUPINFO structure. We call GetStartupInfo to fill the STARTUPINFO structure with default values of the parent process. You MUST fill the STARTUPINFO structure with this call if you intend your code to work under both win9x and NT. After GetStartupInfo call returns, you can modify the members that are important. We copy the handle to the write END of the pipe into hStdOutput and hStdError since we want the child process to use it instead of the default standard output/error handles. We also want to hide the console window of the child process, so we put SW_HIDE value into wShowWidow member. And lastly, we must indicate that hStdOutput, hStdError and wShowWindow members are valid and must be used by specifying the flags STARTF_USESHOWWINDOW and STARTF_USESTDHANDLES in dwFlags member.

        invoke CreateProcess, NULL, ADDR CommandLine, \
                      NULL, NULL, TRUE, NULL, NULL,   \
                      NULL, ADDR startupinfo, ADDR pinfo

We now create the child process with CreateProcess call. Note that the bInheritHandles parameter must be set to TRUE for the pipe handle to work. invoke CloseHandle,hWrite

After we successfully create the child process, we must close the write END of the pipe. Remember that we passed the write handle to the child process via STARTUPINFO structure. If we don't close the write handle from our END, there will be two write ends. And that the pipe will not work. We must close the write handle after CreateProcess but before we read data from the read END of the pipe. .WHILE TRUE invoke RtlZeroMemory,ADDR buffer,1024 invoke ReadFile,hRead,ADDR buffer,1023,ADDR bytesRead,NULL .IF eax==NULL .BREAK .ENDIF invoke SendMessage,hwndEdit,EM_SETSEL,-1,0 invoke SendMessage,hwndEdit,EM_REPLACESEL,FALSE,ADDR buffer .ENDW

Now we are ready to read the data from the standard output of the child process. We will stay in an infinite loop until there are no more data left to read from the pipe. We call RtlZeroMemory to fill the buffer with zeroes then call ReadFile, passing the read handle of the pipe in place of a file handle. Note that we only read a maximum of 1023 bytes since we need the data to be an ASCIIZ string which we can pass on to the edit control.
When ReadFile returns with the data in the buffer, we fill the data into the edit control. However, there is a slight problem here. If we use SetWindowText to put the data into the edit control, the new data will overwrite existing data! We want the data to append to the END of the existing data.
To achieve that goal, we first move the caret to the END of the text in the edit control by sending EM_SETSEL message with wParam=-1. Next, we append the data at that point with EM_REPLACESEL message. invoke CloseHandle,hRead When ReadFile returns NULL, we break out of the loop and close the read handle.

Tutorial 20: Window Subclassing

Overview

Tutorial 22: SuperClassing