Exact meaning of console creation flags
This page is work in progress, will be published soon
Motivation
Win32 CreateProcess function supports a number of process creation flags that affect console windows, namely:
- Default behavior (none of the flags below is specified)
- CREATE_NEW_CONSOLE
- CREATE_NO_WINDOW
- DETACHED_PROCESS
Windows documentation provides limited insight into what these flags actually do. The difference between CREATE_NO_WINDOW and DETACHED_PROCESS is especially obscure. This article explains what to expect when using each flag under different circumstances.
Default (no flags)
If the parent already has a console, or a hidden console buffer, the child process inherits it, and the child's standard I/O handles point to this console or buffer, unless explicitly redirected by the parent. If the parent does not have a console or a hidden buffer, e.g. it is detached or is a GUI process, then the behavior is equivalent to CREATE_NEW_CONSOLE: a new visible console window is created, and the child's standard I/O handles point to it.
CREATE_NEW_CONSOLE
A new visible console window is created for the child process, even if the parent already has one. The size and location of this console and its buffer (but not its code page) can be specified in the fields of the STARTUPINFO structure as described in the documentation. The child's standard input and output will be set to the new console, unless they are explicitly redirected by the parent.
CREATE_NO_WINDOW
This is very similar to CREATE_NEW_CONSOLE, except created console buffer is hidden and no visible window is created. New buffer is created even if the parent already runs in CREATE_NO_WINDOW mode and has its own hidden buffer. Standard I/O handles of the child process will point to this new buffer, unless explicitly redirected by the parent. GetConsoleWindow() returns NULL in the child process, but standard output of the child points to a character device, and GetConsoleBufferInfo() on it returns the buffer's dimensions.
DETACHED_PROCESS
The child process is created without a console, even a hidden one. Standard I/O handles are NULL, unless explicitly set by the parent. GetConsoleWindow() returns NULL, and GetConsoleOutputCP() returns 0. The parent's console is inacessible to the child, even if the parent somehow makes the console buffer handle available to the child.
Using DETACHED_PROCESS flag is rarely a good idea: if the detached process creates child processes of its own without any special flags, their output will not be sent to teh parent's standard output, but will be displayed in newly created consoles, that will flash and disappear.
Summary of combinations
The rules above are applied transitiviely. For example, if a GUI process starts a console child with default flags, which starts a grandchild with CREATE_NO_WINDOW, which starts a grand-grand child with default flags, then the first child will have a visible console, and the grandchild and grand-grandchild will share a hidden console buffer.
As the child and the parent may be created with 4 possible options, this yields 4*4=16 possible combinations. Assuming the parent is launched from a GUI interface, the behavior is as follows:
| Parent flags | Child flags | Visible Consoles | Behavior |
|---|---|---|---|
| Default | Default | 1 | New console created for the parent, the child outputs to the same console. |
| Default | CREATE_NEW_CONSOLE | 2 | Two consoles created, one for the parent, one for the child. |
| Default | CREATE_NO_WINDOW | 1 | New console created for the parent, new hidden buffer created for the child. |
| Default | DETACHED_PROCESS | 1 | New console created for the parent, the child runs without a console or a hidden buffer. |
| CREATE_NEW_CONSOLE | Default | 1 | New console created for the parent, the child outputs to the same console. |
| CREATE_NEW_CONSOLE | CREATE_NEW_CONSOLE | 2 | Two consoles created, one for the parent, one for the child. |
| CREATE_NEW_CONSOLE | CREATE_NO_WINDOW | 1 | New console created for the parent, new hidden buffer created for the child. |
| CREATE_NEW_CONSOLE | DETACHED_PROCESS | 1 | New console created for the parent, the child runs without a console or a hidden buffer. |
| CREATE_NO_WINDOW | Default | 0 | New hidden buffer created for the parent, the child shares that buffer. |
| CREATE_NO_WINDOW | CREATE_NEW_CONSOLE | 1 | New hidden buffer created for the parent, new console created for the child. |
| CREATE_NO_WINDOW | CREATE_NO_WINDOW | 0 | New hidden buffer created for the parent, another one created for the child. |
| CREATE_NO_WINDOW | DETACHED_PROCESS | 0 | New hidden buffer created for the parent, the child runs without a console or a hidden buffer. |
| DETACHED_PROCESS | Default | 1 | The parent runs without a console, new visible console created for the child. |
| DETACHED_PROCESS | CREATE_NEW_CONSOLE | 1 | The parent runs without a console, new visible console created for the child. |
| DETACHED_PROCESS | CREATE_NO_WINDOW | 0 | The parent runs without a console, new hidden buffer created for the child. |
| DETACHED_PROCESS | DETACHED_PROCESS | 0 | The parent and the child run without a console or a hidden buffer. |
Console Creation Rules
Proces of creating a new console is described in the documentation. New console does not inherit the size or the code page of the parent: it behaves as if the application was started anew, and uses either the defaults for the given executable, or system default. Hidden buffer for CREATE_NO_WINDOW is created similarly: it does not have a visible console window associated with it, but its properties can be queried via GetConsoleBufferInfo() call on the standard input or output header. AllocConsole() allocates a new visible console for a process that lacks one, but as far as I know there is no Win32 function for allocating a hidden buffer.
Standard I/O Inheritance Rules
If the parent does not explicitly set the child's standard I/O handles (stdin, stdout, stderr) for the child process, Windows will set them automatically. If a new console or a new hidden buffer is created, the child's standard I/O handles will point to that console/buffer. If the child is detached, the standard handles are set to NULL.
The only case when the child inherits the parent's standard I/O handles is when the parent has a console or a hidden buffer, and the child is created with default flags. Otherwise:
- Parent is DETACHED_PROCSES => no inheritance.
- Child is DETACHED_PROCESS => no inheritance.
- Child creates a new console or a new hidden buffer => no inheritance.
Note that these ruels apply even if the parent's handle is redirected to a file or a pipe by the grandparent. The child will output to the same file or pipe if the parent has a console (hidden or visible), and the child does not create a new one. One would think that presence or lack of a unused console should not matter, but it does.
Standard I/O explicit redirection rules
When creating the child process, the parent can explicitly set the child's standard I/O handles, using STARTUPINFO structure with STARTF_USESTDHANDLES flag. Windows respects that, with one exception: if the parent attempts to redirect the child's otuput to the parent's console, and the child is created with one of CREATE_NEW_CONSOLE, CREATE_NO_WINDOW, DETACHED_PROCESS, then redirection does not work. In the first two cases the redirection is ignored altogether, and in the latter case the child standard handles are be set, but attempts to read/write to them are ignored.
The table below illustrates this with commands executed from cmd.exe console:
| Command line | What it does | Outcome |
|---|---|---|
Exec -w DescribeOutput | Executes child with CREATE_NO_WINDOW, no redirect | No output, child's stdout is a hidden buffer |
Exec -rw DescribeOutput | Executes child with CREATE_NO_WINDOW, redirecting output to the parent's console | No output, child's stdout is a hidden buffer, redirect ignored |
Exec -rw DescribeOutput >foo.txt | Execute child with CREATE_NO_WINDOW, redirecting output to a file | Child's stdout is a file, output text appears in the file |
Feedback
If you have questions or comments, feel free to
send me a note.