CMD piping in unicode

I have two Win 11 systems, a desktop and a laptop.

Their different behaviors have become an issue.

Piping output to a file
- a DIR on the laptop will pipe output in unicode
- a DIR on the laptop will pipe output in ascii

At Code Pages - Win32 apps Microsoft states that the OEM "code pages were originally used for MS-DOS and are still used for console applications.". This was the original behavior of the laptop and CMD chcp verified that the active OEM code page is still 437. The aberration of piping in unicode appeared recently.


Another difference between the two is the CMD window toolbars.
- Right Click on the laptop brings up Properties
- Right Click on the desktop brings up Settings

The laptop used to bring up Settings and it may be that either changing the Default Profile from Windows Powershell to CMD or Default Terminal from Let Windows Decide to Windows Console caused the CMD Toolbar to lose Settings BUT I can find no way back because Settings are needed to change the profile.

I supect the two are related, but have no idea why, am just looking for a way bask to normal behavior on the laptop.

prtajon said:

- a DIR on the laptop will pipe output in unicode
- a DIR on the laptop will pipe output in ascii

Click to expand...

It looks like one of these was meant to say "desktop".

It sounds like you're getting Windows Terminal on one computer and ConHost on the other. On the one running ConHost, just enter Terminal in the Windows search box (or wt in the Run box) to run Terminal and then go to Settings and change the default from Console Host to Windows Terminal.

Thank you Les,

You have solved the toolbar difference, changing to Terminal brought back the Settings on CMD but raised a problem, which is why I changed it in the first place. Terminal ignores the Property / Option to overtype CMD line text by unticking.

Unfortunately, changing to Terminal did not fix the piping problem, the piping destination is still unicode.

Sorry, I'm not quite following. Can you elaborate, preferably with screenshots?

I have taken screenshots which show the CMD commands to take a DIR, a Wordpad of the piped DIR, and the Terminal settings for each. To discriminate in conversation between each I have titled them:
CC=Command Prompt/Console
CT=Command Prompt/Terminal
PC=PowerShell/Console
PT=PowerShell/Terminal
When I changed a configuration in Terminal I Saved, closed the CMD and Wordpad, opened a new CMD to run the DIR and opened those results with Wordpad. Wordpad and WinDbg are the only two that display unicode, rather than an ascii text representation. Win11 deprecated WordPad but so long as en-US is in the same folder it works fine.

Maybe this will work for you too. Before I enabled it, I only got ASCII character when I piped the dir result to a file, ex. `dir *.* -> files.txt`.

In Windows 10/11, there’s a checkbox called “Beta: Use Unicode UTF‑8 for worldwide language support” under:

Control Panel → Region → Administrative → Change system locale…


If this is enabled, Windows uses UTF‑8 (code page 65001) as the default system code page.

@prtajon You're getting output that is consistent with launching Cmd with the /u argument (UTF-16 output mode). Presumably you did not explicitly launch Cmd with /u, so I'm trying to figure out any other way that mode could have been enabled.

Maybe while I'm researching, someone else will jump in with the answer...

@giftwrapped This is an issue where the OP is getting UTF-16 output (i.e. 2 bytes per character). The UTF-8 setting makes no difference in this case.

Edit: ChatGPT's answer:

The only ways cmd.exe produces UTF-16LE output on redirection are:

1. The user explicitly runs cmd /U.
2. There is an AutoRun registry entry that launches cmd /U.
3. A batch/script wraps CMD with /U.

Nothing in regional settings, language, or locale will do this automatically.

giftwrapped: you want unicode and not ascii. I am getting unicode and want ascii.

Les: "Presumably you did not explicitly launch Cmd with /u"

True, this is my launch from Properties
C:\Windows\System32\cmd.exe /user:Administrator /k d:\ini.bat

Your research parallels mine, you have to explicitly request UTF16 to get UTF16 and it is a temporary situation i.e. close the CMD and the next CMD will output OEM/ascii.

One dim light is that terminal without arguments stays open and piping is OEM/ascii.

Unfortunately, terminal is an unsuitable alternative to CMD e.g. Properties/Options such as unticked Insert to set overtype are ignored, Shortcut 'Start in' is ignored, and I can find no /k alternative to initiate a window.

If I set Properties to
C:\Users\admin\AppData\Local\Microsoft\WindowsApps\wt.exe e:\ini.bat
the terminal opens, executes and closes
If I set Properties to
C:\Users\admin\AppData\Local\Microsoft\WindowsApps\wt.exe call e:\ini.bat
the terminal persists because nothing executed due to:
[error 2147942402 (0x80070002) when launching `call e:\ini.bat']
The system cannot find the file specified.

When I changed Advanced Settings from Automatic close to Never close the terminal stays open with:
[process exited with code 0 (0x0000000000)]
You can now close this terminal with Ctrl+D, or press Enter to restart.
Pressing Enter is a Catch 22:
[process exited with code 0 (0x0000000000)]
You can now close this terminal with Ctrl+D, or press Enter to restart.
Ctrl+D closes the terminal.

You've unintentionally invoked UTF-16 encoding. The argument /user:Administrator is not valid. That's being interpreted as /u.

Wunderbar!

Fantastic analysis Les, and curses on whoever wrote the tip that /user:administrator would avoid UAC.

Many thanks for your assistance.

@prtajon, sorry for misunderstanding the problem

No worries giftwrap, help is always welcome.