I'm writing a file manager and need to scan directories and deal with renaming files that may have multibyte characters. I'm working on it locally on Windows/Apache PHP 5.3.8, with the following file names in a directory:

• filename.jpg
• имяфайла.jpg
• file件name.jpg
• פילענאַמע.jpg
• 文件名.jpg

Testing on a live UNIX server woked fine. Testing locally on Windows using glob('./path/*') returns only the first one, filename.jpg.

Using scandir(), the correct number of files is returned at least, but I get names like ?????????.jpg (note: those are regular question marks, not the � character.

I'll end up needing to write a "search" feature to search recursively through the entire tree for filenames matching a pattern or with a certain file extension, and I assumed glob() would be the right tool for that, rather than scan all the files and do the pattern matching and array building in the application code. I'm open to alternate suggestions if need be.

Assuming this was a common problem, I immediately searched Google and Stack Overflow and found nothing even related. Is this a Windows issue? PHP shortcoming? What's the solution: is there anything I can do?

Addendum: Not sure how related this is, but file_exists() is also returning FALSE for these files, passing in the full absolute path (using Notepad++, the php file itself is UTF-8 encoding no BOM). I'm certain the path is correct, as neighboring files without multibyte characters return TRUE.

EDIT: glob() can find a file named filename-äöü.jpg. Previously in my .htaccess file, I had AddDefaultCharset utf-8, which I didn't consider before. filename-äöü.jpg was printing as filename-���.jpg. The only effect removing that htaccess line seemed to have was now that file name prints normally.

I've deleted the .htaccess file completely, and this is my actual test script in it's entirety (I changed a couple of file names from the original post):

print_r(scandir('./uploads/')); 
print_r(glob('./uploads/*'));

Output locally on Windows:

Array
(
    [0] => .
    [1] => ..
    [2] => ??? ?????.jpg
    [3] => ???.jpg
    [4] => ?????????.jpg
    [5] => filename-äöü.jpg
    [6] => filename.jpg
    [7] => test?test.jpg
)
Array
(
    [0] => ./uploads/filename-äöü.jpg
    [1] => ./uploads/filename.jpg
)

Output on remote UNIX server:

Array
(
    [0] => .
    [1] => ..
    [2] => filename-äöü.jpg
    [3] => filename.jpg
    [4] => test이test.jpg
    [5] => имя файла.jpg
    [6] => פילענאַמע.jpg
    [7] => 文件名.jpg
)
Array
(
    [0] => ./uploads/filename-äöü.jpg
    [1] => ./uploads/filename.jpg
    [2] => ./uploads/test이test.jpg
    [3] => ./uploads/имя файла.jpg
    [4] => ./uploads/פילענאַמע.jpg
    [5] => ./uploads/文件名.jpg
)

Since this is a different server, regardless of platform - configuration could be different so I'm not sure what to think, and I can't fully pin it on Windows yet (could be my PHP installation, ini settings, or Apache config). Any ideas?

It looks like the glob() function depends on how your copy of PHP was built and whether it was compiled with a unicode-aware WIN32 API (I don't believe the standard builid is.

Cf. http://www.rooftopsolutions.nl/blog/filesystem-encoding-and-php

Excerpt from comments on the article:

Philippe Verdy 2010-09-26 8:53 am

The output from your PHP installation on Windows is easy to explain : you installed the wrong version of PHP, and used a version not compiled to use the Unicode version of the Win32 API. For this reason, the filesystem calls used by PHP will use the legacy "ANSI" API and so the C/C++ libraries linked with this version of PHP will first try to convert yout UTF-8-encoded PHP string into the local "ANSI" codepage selected in the running environment (see the CHCP command before starting PHP from a command line window)

Your version of Windows is MOST PROBABLY NOT responsible of this weird thing. Actually, this is YOUR version of PHP which is not compiled correctly, and that uses the legacy ANSI version of the Win32 API (for compatibility with the legacy 16-bit versions of Windows 95/98 whose filesystem support in the kernel actually had no direct support for Unicode, but used an internal conversion layer to convert Unicode to the local ANSI codepage before using the actual ANSI version of the API).

Recompile PHP using the compiler option to use the UNICODE version of the Win32 API (which should be the default today, and anyway always the default for PHP installed on a server that will NEVER be Windows 95 or Windows 98...)

Then Windows will be able to store UTF-16 encoded filenames (including on FAT32 volumes, even if, on these volumes, it will also generate an aliased short name in 8.3 format using the filesystem's default codepage, something that can be avoided in NTFS volumes).

All what you describe are problems of PHP (incorrect porting to Windows, or incorrect system version identification at runtime) : reread the README files coming with PHP sources explaining the compilation flags. I really think that the makefile on Windows should be able to configure and autodetect if it really needs to use ONLY the ANSI version of the API. If you are compiling it for a server, make sure that the Configure script will effectively detect the full support of the UNICODE version of the Win32 aPI and will use it when compiling PHP and when selecting the runtime libraries to link.

I use PHP on Windows, correctly compiled, and I absolutely DON'T know the problems you cite in your article.

Let's forget now forever these non-UNICODE versions of the Win32 API (which are using inconsistantly the local ANSI codepage for the Windows graphical UI, and the OEM codepage for the filesystem APIs, the DOS/BIOS-compatible APIs, the Console APIs) : these non-Unicode versions of the APIs are even MUCH slower and more costly than the Unicode versions of the APIs, because they are actually translating the codepage to Unicode before using the core Unicode APIs (the situation on Windows NT-based kernels is exactly the reverse from the situation on versions of Windows based on a virtual DOS extender, such as Windows 95/98/ME).

When you don't use the native version of the API, your API call will pass through a thunking layer that will transcode the strings between Unicode and one of the legacy ANSI or CHCP-selected OEM codepages, or the OEM codepage hinted on the filesystem: this requires additional temporary memory allocation within the non-native version of the Win32 API. This takes additional time to convert things before doing the actual work by calling the native API.

In summary: the PHP binary you install on Windows MUST be different depending on if you compiled it for Windows 95/98/SE (or the old Win16s emulation layer for Windows 3.x, which had a very mimimum support of UTF-8, only to support the Unicode subsets of Unicode used by the ANSI and OEM codapges selected when starting Windows from a DOS extender) or if it was compiled for any other version of Windows based on the NT kernel.

The best proof that this is a problem of PHP and not Windows, is that your weird results will NOT occur in other languages like C#, Javascript, VB, Perl, Ruby... PHP has a very bad history in tracking versions (and too many historical source code quirks and wrong assumptions that should be disabled today, and an inconsistant library that has inherited all those quirks initially made in old versions of PHP for old versions of Windows that are even no longer officially supported, by Microsoft or even by PHP itself !).

In other words : RTFM ! Or download and install a binary version of PHP for Windows precompield with the correct settings : I really think that PHP should distribute Windows binaries already compiled by default for the Unicode version of the Win32 API, and using the Unicode version of the C/C++ libraries : internally the PHP code will convert its UTF-8 strings to UTF-16 before calling the Win32 API, and back from UTF-16 to UTF-8 when retrieving Win32 results, instead of converting PHP's internal UTF-8 strings back/to the local OEM codepage (for the filesystem calls) or the local ANSI codepage (for all other Win32 APIs, including the registry or process).

I was looking at the docs for DuplicateHandle the other day and noticed that DuplicateHandle is able to copy registry key handles (HKEYs). Reading up on this a bit more in the SysInternals book seems to indicate that registry key handles are plain kernel objects, similar to file handles. Yet CloseHandle can't close HKEYs, and RegCloseKey can't close other kinds of kernel objects.

Why the distinction?

It is because only a part of the functionality of the registry is implemented in the kernel. It includes the basic operations (create, delete, read, write, etc.) for working with the local registry keys.

The remaining functions are implemented in the advapi32.dll and work in the user mode:

• Access to a remote registry using RegConnectRegistry
• Access to the HKEY_PERFORMANCE_DATA
• Converting Win32 registry representation to Native representation
• WOW64's registry redirection on 64-bit systems (for 32-bit applications)

The kernel part of the functionality is available through the Native API: NtCreateKey, NtOpenKey, etc. When comparing these functions with the Win32 API it can be seen that the Native API uses the "classical" HANDLE descriptors instead of HKEY.

I want to replace all the popup menus displayed by delphi in the TCustomEdit components like TEdit or TMemo using my own popup menu (which has a lot of more actions). So far I replace the PopUpMenu property of each component manually with my own TPopUpMenu. but I wondering if I can do this without modify this property manually for each component in all my forms.

I want something like a hook to intercept the calls to this system menu and replace for my own menu. is this possible?

In your main form, add the following code. It should apply to all your form's custom control.

TForm2 = class(TForm)
  procedure FormCreate(Sender: TObject);
private
  procedure ActiveControlChanged(Sender: TObject);
end;

implementation

type
  TCustomEditAccess = class(TCustomEdit);
  TCustomGridAccess = class(TCustomGrid);

procedure TForm2.ActiveControlChanged(Sender: TObject);
begin
  if (Screen.ActiveControl is TCustomEdit) and not Assigned(TCustomEditAccess(Screen.ActiveControl).PopupMenu) then
    TCustomEditAccess(Screen.ActiveControl).PopupMenu := MyPopupMenu
  else if (Screen.ActiveControl is TCustomGrid) then
  begin
    TCustomGridAccess(Screen.ActiveControl).ShowEditor;
    if Assigned(TCustomGridAccess(Screen.ActiveControl).InplaceEditor) then
      TCustomEditAccess(TCustomGridAccess(Screen.ActiveControl).InplaceEditor).PopupMenu := MyPopupMenu;
  end;
end;

procedure TForm2.FormCreate(Sender: TObject);
begin
  Screen.OnActiveControlChange := ActiveControlChanged;
end;

It is just a simplified version (in the point of view of coding) of kobik's answer and will also address any TCustomEdit that are created by code or other complex Controls which do not use the Form as Owner.

His instruction on how to determine which CustomEdit popup apply.

Edit : Add Grid InplaceEditor Support

I'v run into an odd problem, my process cannot allocate more than what seems to be slightly below 1 GiB. Windows Task Manager "Mem Usage" column shows values close to 1 GiB when my software gives a bad_alloc exception. Yes, i'v checked that the value passed to memory allocation is sensible. ( no race condition / corruption exists that would make this fail ). Yes, I need all this memory and there is no way around it. ( It's a buffer for images, which cannot be compressed any further )

I'm not trying to allocate the whole 1 GiB memory in one go, there a few allocations around 300 MiB each. Would this cause problems? ( I'll try to see if making more smaller allocations works any better ). Is there some compiler switch or something else that I must set in order to get past 1 GiB? I've seen others complaining about the 2 GiB limit, which would be fine for me.. I just need little bit more :). I'm using VS 2005 with SP1 and i'm running it on a 32 bit XP and it's in C++.

On a 32-bit OS, a process has a 4GB address space in total.

On Windows, half of this is off-limits, so your process has 2GB.

This is 2GB of contiguous memory. But it gets fragmented. Your executable is loaded in at one address, each DLL is loaded at another address, then there's the stack, and heap allocations and so on. So while your process probably has enough free address space, there are no contiguous blocks large enough to fulfill your requests for memory. So making smaller allocations will probably solve it.

If your application is compiled with the LARGEADDRESSAWARE flag, it will be allowed to use as much of the remaining 2GB as Windows can spare. (And the value of that depends on your platform and environment.

• for 32-bit code running on a 64-bit OS, you'll get a full 4-GB address space
• for 32-bit code running on a 32-bit OS without the /3GB boot switch, the flag means nothing at all
• for 32-bit code running on a 32-bit OS with the /3GB boot switch, you'll get 3GB of address space.

So really, setting the flag is always a good idea if your application can handle it (it's basically a capability flag. It tells Windows that we can handle more memory, so if Windows can too, it should just go ahead and give us as large an address space as possible), but you probably can't rely on it having an effect. Unless you're on a 64-bit OS, it's unlikely to buy you much. (The /3GB boot switch is necessary, and it has been known to cause problems with drivers, especially video drivers)

I'm planning to write a Windows app to help myself with some exploratory testing tasks (note taking, data generation, defect logging) and I've got stuck at the early stage of choosing a framework/language. My sole experience is with web development and from what I can see, WinForms, WPF, Silverlight, Swing etc are all simultaneously obsolete and thriving depending on who you ask.

While my main aim is to create the app, obviously I'd like to learn something useful while doing so rather than picking up skills with something that's never going to be seen on a project at work. Which Java or C# frameworks would people recommend learning?

Native Applications

For employment: Well, nowadays most companies (at least most companies from Oman and the UAE, where I live) are slowly migrating to the cloud. However there are still some opportunities for native app development. The most demanding framework nowadays, is, ( no.. not WPF ), it's Windows Forms!

Why plain old Windows Forms instead of the awesome WPF? One reason, legacy apps. Nowadays most companies only start small scale GUI Application projects, mainly Business applications. For that, WPF will be very expensive since they already have a work-force experienced in Windows Forms, and a lot of legacy code, however for WPF they will have to create a new code-base, and that's pretty risky. So the best thing to keep you employed is Windows Forms.

For new projects: However, if by 'worth learning', you mean, new, ambitious and glamorous. Then WPF may be the best choice for you. It depends on what your requirements are, really.

The Cloud

Now, for the cloud. Java FX and Silverlight are both currently head to head. However Java FX may have an edge since it supports a greater number of platforms. But then again, Silverlight has all the power and resources of Microsoft behind it, and it's ideal for Windows Phone development.

Comparison

For a comparison, here's what you get by each toolkit:

Windows Presentation Foundation:

• The power and resources of Microsoft
• Ideal for creating new Desktop Applications
• Eye candy
• Awesome API
• XAML, best way to separate design from logic
• Create Apps for the Cloud (but they only work on Windows with .NET though)
• Windows Phone can run a subset of WPF

Windows Forms:

• Used to possess the power and resources of Microsoft, now WPF has that
• Ideal for maintaining legacy applications
• A well-trained workforce, if you're an entrepreneur
• Pretty mature API
• Supports more platforms than WPF (through Mono)

Java FX:

• Create Apps for the Cloud
• Backed by Oracle
• Pretty nice API
• Cross-platform, runs on most PCs, smart phones are a problem.

Silverlight:

• Create Apps for the Cloud
• Backed by Microsoft
• Pretty awesome API
• XAML
• Cross-platform, runs on Mac and PC, runs on Windows Phone.

GTK#:

• Cross-platform, runs on most PCs, runs on no smart phone.
• Backed by the Open-Source world
• Endorsed by Mono
• Ideal for creating Apps for Gnome.

Swing:

• Cross-platforms, runs on most PCs, smart phones are a problem.
• Pretty mature
• Ideal for creating 2D games, using Java2D

Conclusion

As you say:

While my main aim is to create the app, obviously I'd like to learn something useful while doing so rather than picking up skills with something that's never going to be seen on a project at work.

Well, the frameworks you are most likely to see at work (if you don't for mainstream companies like Microsoft, Oracle, Google etc. ) are Windows Forms and WPF. At least that's what most companies use here. So those are what I recommend. JavaFX and Silverlight also look like they have potential and may be used in the near future.

I'm working on a piece of software which generates assembler code at runtime. For instance, here's a very simple function which generates assembler code for calling the GetCurrentProcess function (for the Win64 ABI):

void genGetCurrentProcess( char *codePtr, FARPROC addressForGetCurrentProcessFunction )
{
#ifdef _WIN64
  // mov rax, addressForGetCurrentProcessFunction
  *codePtr++ = 0x48
  *codePtr++ = 0xB8;
  *((FARPROC *)codePtr)++ = addressForGetCurrentProcessFunction;

  // call rax
  *codePtr++ = 0xFF;
  *codePtr++ = 0xD0;
#else
  // mov eax, addressForGetCurrentProcessfunction
  *codePtr++ = 0xB8;
  *((FARPROC *)codePtr)++ = addressForGetCurrentProcessFunction;

  // call eax
  *codePtr++ = 0xFF;
  *codePtr++ = 0xD0;
#endif
}

Usually I'd use inline assembler, but alas - this doesn't seem to be possible with the 64bit MSVC compilers anymore. While I'm at it - this code should work with MSVC6 up to MSVC10 and also MinGW. There are many more functions like genGetCurrentProcess, they all emit assembler code and many of them get function pointers to be called passed as arguments.

The annoying thing about this is that modifying this code is error-prone and we've got to take care of ABI-specific things manually (for instance, reserving 32 bytes stack space before calling functions for register spilling).

So my question is - can I simplify this code for generating assembler code at runtime? My hope was that I could somehow write the assembler code directly (possibly in an external file which is then assembled using ml/ml64) but it's not clear to me how this would work if some of the bytes in the assembled code are only known at runtime (the addressForGetcurrentProcessFunction value in the above example, for instance). Maybe it's possible to assemble some code but assign 'labels' to certain locations in the code so that I can easily modify the code at runtime and then copy it into my buffer?

Take a look at asmjit. It is a C++ library for runtime code-generation. Supports x64 and probably most of the existing extensions (FPU, MMX, 3dNow, SSE, SSE2, SSE3, SSE4). Its interface resembles assembly syntax and it encodes the instructions correctly for you.

I would like to detect when the form is going to be maximized to save certain settings (not related to the size nor position) and modify the size and position a little bit. Is there an universal way to do it ? I've tried to catch the WM_SYSCOMMAND message like in this article. It works well for maximization from menu, by maximize button, but it's not fired when I press the WIN + UP keystroke. Does anyone know an universal way how to catch the maximization event including the case with WIN + UP keystroke ?

Thanks

You can use the WM_GETMINMAXINFO message to save the state of the window and then use the WMSize message to check if the window was maximized.

in you form declare the mesage handler like so :

procedure WMSize(var Msg: TMessage); message WM_SIZE;

And handle like this :

procedure TForm57.WMSize(var Msg: TMessage);
begin
  if Msg.WParam  = SIZE_MAXIMIZED then
    ShowMessage('Maximized');    
end;

I'm curious if allocating a buffer on the stack is required to have correct alignment for any type, similar to how malloc works, or if I would be forced to use something like std::aligned_storage.

Consider the following block of code:

typedef enum _KEY_VALUE_INFORMATION_CLASS {
    KeyValueBasicInformation            = 0,
    // Others
} KEY_VALUE_INFORMATION_CLASS;

typedef struct _KEY_VALUE_BASIC_INFORMATION {
    ULONG TitleIndex;
    ULONG Type;
    ULONG NameLength;
    WCHAR Name[1];
} KEY_VALUE_BASIC_INFORMATION, *PKEY_VALUE_BASIC_INFORMATION;

std::vector<std::wstring> RegistryKey::EnumerateValueNames() const
{
    std::vector<std::wstring> result;
    ULONG index = 0;
    const ULONG valueNameStructSize = 16384 * sizeof(wchar_t) +
        sizeof(KEY_VALUE_BASIC_INFORMATION);

    // Stack buffer here
    unsigned char buff[valueNameStructSize];
    // Casted here
    KEY_VALUE_BASIC_INFORMATION const* basicValueInformation =
        reinterpret_cast<KEY_VALUE_BASIC_INFORMATION const*>(buff);
    for(;;)
    {
        ULONG resultLength;
        NTSTATUS errorCheck = PNtEnumerateValueKeyFunc(
            hKey_,
            index++,
            KeyValueBasicInformation,
            buff,
            valueNameStructSize,
            &resultLength);
        if (NT_SUCCESS(errorCheck))
        {
            result.emplace_back(std::wstring(basicValueInformation->Name,
                basicValueInformation->NameLength / sizeof(wchar_t)));
        }
        else if (errorCheck == STATUS_NO_MORE_ENTRIES)
        {
            break;
        }
        else
        {
            Win32Exception::ThrowFromNtError(errorCheck);
        }
    }
    return result;
}

Note how the value buff is a character buffer put on the stack, sized to hold a given maximum amount of data. However, I'm concerned that the cast required to interpret the buffer as a string may cause an alignment fault if this code were to be ported to another (say ARM or IA64) platform.

EDIT: If anyone's curious, I redid this in terms of std::aligned_storage and std::alignment_of:

std::vector<std::wstring> RegistryKey::EnumerateValueNames() const
{
    std::vector<std::wstring> result;
    ULONG index = 0;
    const ULONG valueNameStructSize = 16384 * sizeof(wchar_t) +
        sizeof(KEY_VALUE_BASIC_INFORMATION);
    std::aligned_storage<valueNameStructSize,
        std::alignment_of<KEY_VALUE_BASIC_INFORMATION>::value>::type buff;
    auto basicValueInformation =
        reinterpret_cast<KEY_VALUE_BASIC_INFORMATION*>(&buff);
    for(;;)
    {
        ULONG resultLength;
        NTSTATUS errorCheck = PNtEnumerateValueKeyFunc(
            hKey_,
            index++,
            KeyValueBasicInformation,
            basicValueInformation,
            valueNameStructSize,
            &resultLength);
        if (NT_SUCCESS(errorCheck))
        {
            result.emplace_back(std::wstring(basicValueInformation->Name,
                basicValueInformation->NameLength / sizeof(wchar_t)));
        }
        else if (errorCheck == STATUS_NO_MORE_ENTRIES)
        {
            break;
        }
        else
        {
            Win32Exception::ThrowFromNtError(errorCheck);
        }
    }
    return std::move(result);
}

The standard makes no requirements on the alignment of automatic variables (or variables with static storage for that matter), other than the compiler must make sure that accessing them works.

• C++03 3.9/5 Types

Object types have alignment requirements (3.9.1, 3.9.2). The alignment of a complete object type is an implementation-defined integer value representing a number of bytes; an object is allocated at an address that meets the alignment requirements of its object type

Note: "object type" here means a type that's not a function, reference or void type, (i.e., it applies to unsigned char).

One way to get an aligned buffer might be to declare buff like so:

KEY_VALUE_BASIC_INFORMATION buff[valueNameStructSize/sizeof(KEY_VALUE_BASIC_INFORMATION) + 1];

And you'll be able to get rid of the reinterpret_cast<> to boot.

On my OS X machine, the following line gives me a nice and easy way to track the state of my loops:

for (int index = 0; index < 100; index++)
    for (int subIndex = index; subIndex < 100; subIndex++)
        System.out.print("\r" + index + "/" + subIndex + "       ");

But when I try to run the same thing on windows, it prints out newlines instead of a carriage return. How can I achieve the same simple method of tracking the process on windows?

I had the statement and it worked in the command prompt

System.out.println("This is Java"+'\r'+"That");

and gives me output as

That is Java

That means it works perfectly.

Note: I run it in Windows 7 with JDK 7 and simple notepad.

It is the problem of eclipse, it will take \r as a new line character and will print

This is Java
That

as output

So I have used a number of tools to create msi installers for my stuff, including things like WiX and a couple of the many GUI's around.

However one thing that I have never really worked out is what part does Windows Installer itself actually do, and where do these tools start and end? For that matter, what exactly is the msi technically, and why is it no one (I couldn't even find information on how it could be done in theory, like if its actually just some kind of DLL type thing that implements a simple interface) create an msi themselves, without using one of these tools to make it for them?

Some years ago I asked me myself the questions like "What is MSI file?", "How one can create or decode it?", "Why MSI database structure looks so strange?". So I answered for me on the questions. If you have an interest I can share the knowledge with you.

About the history. Windows Installer technology was introduced by Microsoft Office installer team during developing of Office 2000 setup. Before that Office 97 setup was STF based. The STF file consist from two tables: one with general information which can be compared with Properties table of MSI and another table which described the order of execution of different installation steps. Three main starting modes was supported: Installation (Administrative installation is sub-mode), Remove and Maintains. Office Software was going more and more complex and Microsoft wanted to make the setups more stable. See here for some more information.

Last years of the 20-century was the COM and COM+ time at Microsoft. The format of WinWord documents was COM Structured Storage too. So the format of the MSI file was chosen also as the structured storage. In general one wanted only to save some separate information like tables in one file. It's important that some tables will be modified during the installation. So one want be sure that the whole MSI file will be not corrupted in case of failed installation. The structured storage provided minimal support for the case, so the format will used since the time. The modified MSI files will be saved in the %SystemRoot%\Installer folder.

If you open MSI file with respect of Orca tool and export all tables in the files you will have exact the same information set which you have in MSI. You can modify the text tiles and then import the files back. If you would get some empty MSI file and import the tables you will create new Windows Installer setup.

Windows SDK has a list of Scripts which you can find in the C:\Program Files\Microsoft SDKs\Windows\v7.1\Samples\sysmgmt\msi\scripts folder. You can use the scripts to create empty MSI and to import the tables in MSI. So you can use the scripts instead of WiX. The WiX uses XML format which makes the input information more readable as idt-files (tables exported by Orca) and more easy to maintain.

For better understanding I wrote some years ago some small utilities which create the empty MSI files without Windows Installer API and which just used COM Structured Storage API. Additionally I created utility which decodes full information from MSI files on the low-level (also without usage of Windows Installer API). So I am sure that MSI files are really not more as I described above.

I see that you are C/C++ developer. If it would be interesting for you you can play with the C program which create empty MSI.

#define STRICT
#define _WIN32_WINNT 0x501
#define COBJMACROS

#include <stdio.h>
#include <windows.h>
#pragma warning (disable: 4201)
#include <ShLwApi.h>    // for wnsprintf
#pragma warning (default: 4201)
#include <malloc.h>     // for _alloca
#include <lmerr.h>
#include <tchar.h>

#define ARRAY_SIZE(arr)     (sizeof(arr)/sizeof(arr[0]))
#define CONST_STR_LEN(s)    (ARRAY_SIZE(s) - 1)

#pragma comment (lib, "ole32.lib")
#pragma comment (lib, "ShLwApi.lib")

#define MIDL_DEFINE_GUID(type,name,l,w1,w2,b1,b2,b3,b4,b5,b6,b7,b8) const type name = {l,w1,w2,{b1,b2,b3,b4,b5,b6,b7,b8}}

MIDL_DEFINE_GUID (CLSID, CLSID_MsiTransform, 0x000c1082, 0x0000, 0x0000, 0xc0,0x00,0x00,0x00,0x00,0x00,0x00,0x46);  //.mst
MIDL_DEFINE_GUID (CLSID, CLSID_MsiDatabase,  0x000c1084, 0x0000, 0x0000, 0xc0,0x00,0x00,0x00,0x00,0x00,0x00,0x46);  //.msi, .msm
MIDL_DEFINE_GUID (CLSID, CLSID_MsiPatch,     0x000c1086, 0x0000, 0x0000, 0xc0,0x00,0x00,0x00,0x00,0x00,0x00,0x46);  //.msp

// This function do almost the same as Base64 encoding used for example in MIME (see 6.8 in http://www.ietf.org/rfc/rfc2045.txt).
// Base64 convert codes from 0 till 63 (0x3F) to the corresponding character from the array 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/'
// This function convert it to the corresponding character from the another array '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz._'
static BYTE MsiBase64Encode (BYTE x)
{
    // 0-0x3F converted to '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz._'
    // all other values higher as 0x3F converted also to '_'
    if (x < 10)
        return x + '0';             // 0-9 (0x0-0x9) -> '0123456789'
    else if (x < (10+26))
        return x - 10 + 'A';        // 10-35 (0xA-0x23) -> 'ABCDEFGHIJKLMNOPQRSTUVWXYZ'
    else if (x < (10+26+26))
        return x - 10 - 26 + 'a';   // 36-61 (0x24-0x3D) -> 'abcdefghijklmnopqrstuvwxyz'
    else if (x == (10+26+26))       // 62 (0x3E) -> '.'
        return '.';
    else
        return '_';                 // 63-0xffffffff (0x3F-0xFFFFFFFF) -> '_'
}

#pragma warning (disable: 4706)
static UINT DecodeStreamName (LPWSTR pszInStreamName, LPWSTR pszOutStreamName)
{
    WCHAR ch;
    DWORD count = 0;

    while ((ch = *pszInStreamName++)) {
        if ((ch >= 0x3800) && (ch < 0x4840)) {
            // a part of Unicode charecterd used with CJK Unified Ideographs Extension A. (added with Unicode 3.0) used by
            // Windows Installer for encoding one or two ANSI characters. This subset of Unicode characters are not currently
            // used nether in "MS PMincho" or "MS PGothic" font nor in "Arial Unicode MS"
            if (ch >= 0x4800)   // 0x4800 - 0x483F
                // only one charecter can be decoded
                ch = (WCHAR) MsiBase64Encode ((BYTE)(ch - 0x4800));
            else {              // 0x3800 - 0x383F
                // the value contains two characters
                ch -= 0x3800;
                *pszOutStreamName++ = (WCHAR) MsiBase64Encode ((BYTE)(ch & 0x3f));
                count++;
                ch = (WCHAR) MsiBase64Encode ((BYTE)((ch >> 6) & 0x3f));
            }
        }
        // all characters lower as 0x3800 or higher as 0x4840 will be saved without changes

        *pszOutStreamName++ = ch;
        count++;
    }
    *pszOutStreamName = L'\0';

    return count;
}
#pragma warning (default: 4706)

#define INVALID_DECODING_RESULT ((BYTE)(-1))
// This function do almost the same as Base64 decoding used for example in MIME (see 6.8 in http://www.ietf.org/rfc/rfc2045.txt).
// Base64 convert character from the array 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/' to the corresponding codes from 0 till 63 (0x3F)
// This function convert character from the another array '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz._' to it to 0 till 63 (0x3F)
static BYTE MsiBase64Decode (BYTE ch)
// returns values 0 till 0x3F or 0xFF in the case of an error
{
    // only '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz._' are allowed and converted to 0-0x3F
    if ((ch>=L'0') && (ch<=L'9'))   // '0123456789' -> 0-9  (0x0-0x9)
        return ch-L'0';
    else if ((ch>=L'A') && (ch<=L'Z'))   // 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' -> 10-35 (26 chars) - (0xA-0x23)
        return ch-'A'+10;
    else if ((ch>=L'a') && (ch<=L'z'))   // 'abcdefghijklmnopqrstuvwxyz' -> 36-61 (26 chars) - (0x24-0x3D)
        return ch-L'a'+10+26;
    else if (ch==L'.')
        return 10+26+26;        // '.' -> 62 (0x3E)
    else if (ch==L'_')
        return 10+26+26+1;      // '_' -> 63 (0x3F) - 6 bits
    else
        return INVALID_DECODING_RESULT; // other -> -1 (0xFF)
}

#define MAX_STREAM_NAME 0x1f

static void EncodeStreamName (BOOL bTable, LPCWSTR pszInStreamName, LPWSTR pszOutStreamName, UINT cchOutStreamName)
{
    LPWSTR pszCurrentOut = pszOutStreamName;

    if (bTable) {
         *pszCurrentOut++ = 0x4840;
         cchOutStreamName--;
    }

    while (cchOutStreamName--) {
        WCHAR ch = *pszInStreamName++;

        if (ch && (ch < 0x80) && (MsiBase64Decode((BYTE)ch) <= 0x3F)) {
            WCHAR chNext = *pszInStreamName;

            // MsiBase64Decode() convert any "standard" character '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz._' to 0-0x3F.
            // One can pack two charecters together in 0-0xFFF. To do so, one needs convert the first one with respect of MsiBase64Decode(),
            // convert the next character also with respect MsiBase64Decode() and shift it 6 bits on the left. Two characters together
            // produce a value from 0 till 0xFFF. We add 0x3800 to the result. We receive a value between 0x3800 and 0x47FF
            if (chNext && (chNext < 0x80) && (MsiBase64Decode((BYTE)chNext) <= 0x3F)) {
                ch = (WCHAR)(MsiBase64Decode((BYTE)ch) + 0x3800 + (MsiBase64Decode((BYTE)chNext)<<6));
                pszInStreamName++;
            }
            else
                ch = MsiBase64Decode((BYTE)ch) + 0x4800;
        }
        *pszCurrentOut++ = ch;

        if (!ch)
            break;
    }
}

enum tagStringIds {
    IDS_PROPERTY = 1,           // Property
    IDS_VALUE,                  // Value
    IDS_MANUFACTURER,           // Manufacturer
    IDS_MANUFACTURER_VALUE,     // "OK soft GmbH"
    IDS_PRODUCT_LANGUAGE,       // ProductLanguage
    IDS_PRODUCT_LANGUAGE_VALUE, // 1033
    IDS_PRODUCT_VERSION,        // ProductVersion
    IDS_PRODUCT_VERSION_VALUE,  // 1.0
    IDS_PRODUCT_NAME,           // ProductName
    IDS_PRODUCT_NAME_VALUE,     // "Trust to User (T2U) Service"
    IDS_PRODUCT_CODE,           // ProductCode
    IDS_PRODUCT_CODE_VALUE,     // {B2F99E7E-6C86-4D60-A6D1-F81CAC15B0F7}
    IDS_UPGRADE_CODE,           // UpgradeCode
    IDS_UPGRADE_CODE_VALUE      // {EE115A5D-D05A-465F-B077-F28CCDB20ECB}
};

//struct _StringPool {
//    WORD wLength;
//    WORD wRefcnt;
//} *pStringPool = NULL;
struct StrintgTable {
    UINT uId;
    UINT cRefcnt;
} g_StrintgTable[] = {
    {IDS_PROPERTY,               4},
    {IDS_VALUE,                  1},
    {IDS_MANUFACTURER,           1},
    {IDS_MANUFACTURER_VALUE,     1},
    {IDS_PRODUCT_LANGUAGE,       1},
    {IDS_PRODUCT_LANGUAGE_VALUE, 1},
    {IDS_PRODUCT_VERSION,        1},
    {IDS_PRODUCT_VERSION_VALUE,  1},
    {IDS_PRODUCT_NAME,           1},
    {IDS_PRODUCT_NAME_VALUE,     1},
    {IDS_PRODUCT_CODE,           1},
    {IDS_PRODUCT_CODE_VALUE,     1},
    {IDS_UPGRADE_CODE,           1},
    {IDS_UPGRADE_CODE_VALUE,     1}
};

//Id:   13  Refcnt:    4  String: Property
//Id:    1  Refcnt:    1  String: Value
//Id:    2  Refcnt:    1  String: {EE115A5D-D05A-465F-B077-F28CCDB20ECB}
//Id:    3  Refcnt:    1  String: ProductLanguage
//Id:    4  Refcnt:    1  String: UpgradeCode
//Id:    5  Refcnt:    1  String: {B2F99E7E-6C86-4D60-A6D1-F81CAC15B0F7}
//Id:    6  Refcnt:    1  String: 1.0
//Id:    7  Refcnt:    1  String: ProductCode
//Id:    8  Refcnt:    1  String: ProductVersion
//Id:    9  Refcnt:    1  String: OK soft GmbH
//Id:   10  Refcnt:    1  String: Trust to User (T2U) Service
//Id:   11  Refcnt:    1  String: Manufacturer
//Id:   12  Refcnt:    1  String: ProductName
//Id:   14  Refcnt:    1  String: 1033

UINT g_Tabeles[] = {IDS_PROPERTY};
//uStatus = ExecuteQueryWirhTwoStringParameters (hView, TEXT("Manufacturer"), TEXT("OK soft GmbH"));
//uStatus = ExecuteQueryWirhTwoStringParameters (hView, TEXT("ProductLanguage"), TEXT("1033"));
//uStatus = ExecuteQueryWirhTwoStringParameters (hView, TEXT("ProductVersion"), TEXT("1.0"));
//uStatus = ExecuteQueryWirhTwoStringParameters (hView, TEXT("ProductCode"), TEXT("{B2F99E7E-6C86-4D60-A6D1-F81CAC15B0F7}"));
//uStatus = ExecuteQueryWirhTwoStringParameters (hView, TEXT("ProductName"), TEXT("Trust to User (T2U) Service"));

int main()
{
    HRESULT hr;
    LPCWSTR pszFilename = L"Empty.msi";
    IStorage *pStg = NULL;
    IStream *pStm = NULL;
    IPropertySetStorage *pPropSetStg = NULL;
    IPropertyStorage *pPropStg = NULL;
    WCHAR szOutStreamName[64];
    WORD wCodePage, wStringIdSize;
    ULONG cbWritten;
    PROPSPEC rgpspec[8] = {
        {PRSPEC_PROPID, PIDSI_TITLE},
        {PRSPEC_PROPID, PIDSI_SUBJECT},
        {PRSPEC_PROPID, PIDSI_AUTHOR},
        {PRSPEC_PROPID, PIDSI_KEYWORDS},
        {PRSPEC_PROPID, PIDSI_TEMPLATE},
        {PRSPEC_PROPID, PIDSI_REVNUMBER},
        {PRSPEC_PROPID, PIDSI_PAGECOUNT},
        {PRSPEC_PROPID, PIDSI_WORDCOUNT}
    };
    PROPVARIANT rgpropvar[8];
    PROPSPEC pspec;
    PROPVARIANT propvar = {0};

    hr = StgCreateStorageEx (pszFilename,
                             STGM_CREATE | STGM_DIRECT | STGM_READWRITE | STGM_SHARE_EXCLUSIVE,
                             STGFMT_DOCFILE,
                             0,
                             NULL,
                             NULL,
                             &IID_IStorage,
                             &pStg);
    if (FAILED(hr))
        return hr;

    hr = IStorage_SetClass (pStg, &CLSID_MsiDatabase);
    // file has 1536 bytes (512*3)

    hr = IStorage_QueryInterface (pStg, &IID_IPropertySetStorage, &pPropSetStg);
    hr = IPropertySetStorage_Create (pPropSetStg, &FMTID_SummaryInformation, NULL, PROPSETFLAG_ANSI,
                                     STGM_CREATE | STGM_READWRITE | STGM_SHARE_EXCLUSIVE, &pPropStg);

    pspec.propid = PRSPEC_PROPID;
    pspec.ulKind = PID_CODEPAGE;
    PropVariantInit (&propvar);
    propvar.vt = VT_I2;
    propvar.iVal = 1252;
    hr = IPropertyStorage_WriteMultiple (pPropStg, 1, &pspec, &propvar, 0);

    PropVariantInit (&rgpropvar[0]);
    rgpropvar[0].vt = VT_LPSTR;
    rgpropvar[0].pszVal = "Installation Database";
    PropVariantInit (&rgpropvar[1]);
    rgpropvar[1].vt = VT_LPSTR;
    rgpropvar[1].pszVal = "Trust To User (T2U) Service";
    PropVariantInit (&rgpropvar[2]);
    rgpropvar[2].vt = VT_LPSTR;
    rgpropvar[2].pszVal = "OK soft GmbH";
    PropVariantInit (&rgpropvar[3]);
    rgpropvar[3].vt = VT_LPSTR;
    rgpropvar[3].pszVal = "Installer,MSI,Database";
    PropVariantInit (&rgpropvar[4]);
    rgpropvar[4].vt = VT_LPSTR;
    rgpropvar[4].pszVal = "Intel;1033";
    PropVariantInit (&rgpropvar[5]);
    rgpropvar[5].vt = VT_LPSTR;
    rgpropvar[5].pszVal = "{B2F99E7E-6C86-4D60-A6D1-F81CAC15B0F7}";
    PropVariantInit (&rgpropvar[6]);
    rgpropvar[6].vt = VT_I4;
    rgpropvar[6].lVal = 110;
    PropVariantInit (&rgpropvar[7]);
    rgpropvar[7].vt = VT_I4;
    rgpropvar[7].pszVal = 0;
    hr = IPropertyStorage_WriteMultiple (pPropStg, ARRAY_SIZE(rgpspec), rgpspec, rgpropvar, PIDSI_TITLE);   // PID_FIRST_USABLE

    hr = IPropertyStorage_Commit (pPropStg, 0);

    EncodeStreamName (TRUE, L"_Tables", szOutStreamName, ARRAY_SIZE(szOutStreamName));
    hr = IStorage_CreateStream (pStg, szOutStreamName,
                                STGM_CREATE | STGM_READWRITE | STGM_SHARE_EXCLUSIVE,
                                0, 0, &pStm);
    for (i=0; i<ARRAY_SIZE(g_Tabeles); i++) {
        WORD w = g_Tabeles[i];
        hr = IStream_Write (pStm, (LPCVOID)&w, sizeof(WORD), &cbWritten);
    }
    IStream_Release (pStm);
    // file has 1536 bytes (512*3)

    EncodeStreamName (TRUE, L"_StringData", szOutStreamName, ARRAY_SIZE(szOutStreamName));
    hr = IStorage_CreateStream (pStg, szOutStreamName,
                                STGM_CREATE | STGM_READWRITE | STGM_SHARE_EXCLUSIVE,
                                0, 0, &pStm);
    IStream_Release (pStm);

    EncodeStreamName (TRUE, L"_StringPool", szOutStreamName, ARRAY_SIZE(szOutStreamName));
    hr = IStorage_CreateStream (pStg, szOutStreamName,
                                STGM_CREATE | STGM_READWRITE | STGM_SHARE_EXCLUSIVE,
                                0, 0, &pStm);

    wCodePage = 1252;
    wStringIdSize = 0;  // 2 bytes
    hr = IStream_Write (pStm, (LPCVOID)&wCodePage, sizeof(WORD), &cbWritten);
    hr = IStream_Write (pStm, (LPCVOID)&wStringIdSize, sizeof(WORD), &cbWritten);

    IStream_Release (pStm);
    // 2560 bytes (512*5)

    IPropertyStorage_Release (pPropStg);
    IPropertySetStorage_Release (pPropSetStg);

    IStorage_Release (pStg);

    return hr;
}

The code of the program which dumps MSI is more long and I don't see that it is really needed for you.

The utility which use Windows Installer API and create an empty MSI is below. It create more full valid MSI in sense of MSI-Validation :

#include <windows.h>
#include <Msi.h>
#include <MsiQuery.h>
#pragma warning (disable: 4201)
#include <ShLwApi.h>
#pragma warning (default: 4201)

#pragma comment (lib, "Msi.lib")
#pragma comment (lib, "ShLwApi.lib")

#define ARRAY_SIZE(ar)   (sizeof(ar)/sizeof(ar[0]))
#define CONST_STR_LEN(s) (ARRAY_SIZE(s) - 1)

UINT ExecuteSimpleMsiQuery (MSIHANDLE hDatabase, LPCTSTR pszQuery)
{
    UINT uStatus = ERROR_INVALID_DATA;
    MSIHANDLE hView = (MSIHANDLE)0;

    __try {
        uStatus = MsiDatabaseOpenView (hDatabase, pszQuery, &hView);
        if (uStatus != NO_ERROR) __leave;
        uStatus = MsiViewExecute (hView, (MSIHANDLE)0);
        if (uStatus != NO_ERROR) __leave;
        uStatus = MsiViewClose(hView);
    }
    __finally {
        if (hView != (MSIHANDLE)0)
            MsiCloseHandle (hView);
    }

    return uStatus;
}

UINT ExecuteQueryWirhTwoStringParameters (MSIHANDLE hView, LPCTSTR pszStr1, LPCTSTR pszStr2)
{
    UINT uStatus = ERROR_INVALID_DATA;
    MSIHANDLE hRec = (MSIHANDLE)0;

    __try {
        hRec = MsiCreateRecord(2);
        MsiRecordSetString (hRec, 1, pszStr1);
        MsiRecordSetString (hRec, 2, pszStr2);

        uStatus = MsiViewExecute (hView, hRec);

        // prepair for the next call of MsiViewExecute
        uStatus = MsiViewClose(hView);
    }
    __finally {
        if (hRec != (MSIHANDLE)0)
           uStatus = MsiCloseHandle (hRec);
    }

    return uStatus;
}

void main()
{
    LPCTSTR pszMsiName = TEXT("Empty.msi");
    MSIHANDLE hDatabase = (MSIHANDLE)0, hSummaryInfo = (MSIHANDLE)0, hView = (MSIHANDLE)0;
    UINT uiUpdateCount;
    UINT uStatus;
    HANDLE hFile = INVALID_HANDLE_VALUE;
    BOOL bSuccess;
    char szPropertyValues[] =
        "Property\tValue\r\n"
        "s72\tl0\r\n"
        "Property\tProperty\r\n"
        "Manufacturer\tOK soft GmbH\r\n"
        "ProductCode\t{B2F99E7E-6C86-4D60-A6D1-F81CAC15B0F7}\r\n"
        "ProductLanguage\t1031\r\n"
        "ProductName\tTrust to User (T2U) Service\r\n"
        "ProductVersion\t1.0\r\n"
        "UpgradeCode\t{EE115A5D-D05A-465F-B077-F28CCDB20ECB}\r\n";
    DWORD cbNumberOfBytesWritten;
    char szBuffer[128];

    __try {
        UINT i, cMaxProperty=0xFFFFFF;

        // Create empty database. Inspite of it is empty MSI file has 2560 bytes
        uStatus = MsiOpenDatabase (pszMsiName, MSIDBOPEN_CREATE, &hDatabase);
        if (uStatus != NO_ERROR) __leave;

        uiUpdateCount = 9;
        uStatus = MsiGetSummaryInformation (hDatabase, NULL, uiUpdateCount, &hSummaryInfo);
        if (uStatus != NO_ERROR) __leave;

        // PID_CODEPAGE is optional
        uStatus = MsiSummaryInfoSetProperty (hSummaryInfo, PID_CODEPAGE,     VT_I2, 1252, NULL, NULL);
//C:\Oleg\Win32.new\MSI\CreateEmptyMsi>msiinfo C:\Oleg\Win32.new\MSI\CreateEmptyMsi\Empty.msi
//
//Class Id for the MSI storage is {000C1084-0000-0000-C000-000000000046}
//
//Error 1627. Unable to display summary information. System does not support the codepage of the Summary Information Stream (codepage = '1200')


        // VT_LPSTR MUST be used and not VT_LPWSTR.
        // If one use MsiSummaryInfoSetPropertyW(..., VT_LPWSTR, 9, NULL, L"string"); one receive 1629 ERROR - ERROR_DATATYPE_MISMATCH
        uStatus = MsiSummaryInfoSetPropertyW (hSummaryInfo, PIDSI_TITLE,     VT_LPSTR, 0, NULL, L"Installation Database");

        // PIDSI_SUBJECT and PIDSI_AUTHOR are optional
        uStatus = MsiSummaryInfoSetPropertyW (hSummaryInfo, PIDSI_SUBJECT,   VT_LPSTR, 0, NULL, L"Trust To User (T2U) Service");
        uStatus = MsiSummaryInfoSetPropertyW (hSummaryInfo, PIDSI_AUTHOR,    VT_LPSTR, 0, NULL, L"OK soft GmbH");
        uStatus = MsiSummaryInfoSetPropertyW (hSummaryInfo, PIDSI_KEYWORDS,  VT_LPSTR, 0, NULL, L"Installer,MSI,Database");

        uStatus = MsiSummaryInfoSetPropertyW (hSummaryInfo, PIDSI_TEMPLATE,  VT_LPSTR, 0, NULL, L"Intel;1033");
        uStatus = MsiSummaryInfoSetPropertyW (hSummaryInfo, PIDSI_REVNUMBER, VT_LPSTR, 0, NULL, L"{B2F99E7E-6C86-4D60-A6D1-F81CAC15B0F7}");

        uStatus = MsiSummaryInfoSetProperty (hSummaryInfo, PIDSI_PAGECOUNT,  VT_I4, 110, NULL, NULL);
        uStatus = MsiSummaryInfoSetProperty (hSummaryInfo, PIDSI_WORDCOUNT,  VT_I4, 0, NULL, NULL);
        uStatus = MsiSummaryInfoPersist (hSummaryInfo);
        // if we commit database here we receive 3072 large MSI file


        uStatus = ExecuteSimpleMsiQuery (hDatabase, TEXT("CREATE TABLE `Property` (`Property` CHAR(72) NOT NULL, `Value` CHAR(0) NOT NULL LOCALIZABLE PRIMARY KEY `Property`)"));
        if (uStatus != NO_ERROR) __leave;

        uStatus = MsiDatabaseOpenView (hDatabase, TEXT("INSERT INTO `Property` (`Property`, `Value`) VALUES (?, ?)"), &hView);
        if (uStatus != NO_ERROR) __leave;
        uStatus = ExecuteQueryWirhTwoStringParameters (hView, TEXT("Manufacturer"), TEXT("OK soft GmbH"));
        uStatus = ExecuteQueryWirhTwoStringParameters (hView, TEXT("ProductLanguage"), TEXT("1033"));
        uStatus = ExecuteQueryWirhTwoStringParameters (hView, TEXT("ProductVersion"), TEXT("1.0"));
        uStatus = ExecuteQueryWirhTwoStringParameters (hView, TEXT("ProductCode"), TEXT("{B2F99E7E-6C86-4D60-A6D1-F81CAC15B0F7}"));
        uStatus = ExecuteQueryWirhTwoStringParameters (hView, TEXT("ProductName"), TEXT("Trust to User (T2U) Service"));
        uStatus = ExecuteQueryWirhTwoStringParameters (hView, TEXT("UpgradeCode"), TEXT("{EE115A5D-D05A-465F-B077-F28CCDB20ECB}"));

        uStatus = ExecuteSimpleMsiQuery (hDatabase, TEXT("CREATE TABLE `_Validation` (`Table` CHAR(32) NOT NULL, `Column` CHAR(32) NOT NULL, `Nullable` CHAR(4) NOT NULL, `MinValue` LONG, `MaxValue` LONG, `KeyTable` CHAR(255), `KeyColumn` INT, `Category` CHAR(32), `Set` CHAR(255), `Description` CHAR(255) PRIMARY KEY `Table`, `Column`)"));
        if (uStatus != NO_ERROR) __leave;
        uStatus = ExecuteSimpleMsiQuery (hDatabase, TEXT("INSERT INTO `_Validation` (`Table`, `Column`, `Nullable`, `MinValue`, `MaxValue`, `KeyTable`, `KeyColumn`, `Category`, `Set`, `Description`) VALUES (")
                                                    TEXT("'_Validation', 'Category', 'Y', NULL, NULL, NULL, NULL, NULL, 'Text;Formatted;Template;Condition;Guid;Path;Version;Language;Identifier;Binary;UpperCase;LowerCase;Filename;Paths;AnyPath;WildCardFilename;RegPath;KeyFormatted;CustomSource;Property;Cabinet;Shortcut;URL', 'String category')"));
        uStatus = ExecuteSimpleMsiQuery (hDatabase, TEXT("INSERT INTO `_Validation` (`Table`, `Column`, `Nullable`, `MinValue`, `MaxValue`, `KeyTable`, `KeyColumn`, `Category`, `Set`, `Description`) VALUES (")
                                                    TEXT("'_Validation', 'Column', 'N', NULL, NULL, NULL, NULL, 'Identifier', NULL, 'Name of column')"));
        uStatus = ExecuteSimpleMsiQuery (hDatabase, TEXT("INSERT INTO `_Validation` (`Table`, `Column`, `Nullable`, `MinValue`, `MaxValue`, `KeyTable`, `KeyColumn`, `Category`, `Set`, `Description`) VALUES (")
                                                    TEXT("'_Validation', 'Description', 'Y', NULL, NULL, NULL, NULL, 'Text', NULL, 'Description of column')"));
        uStatus = ExecuteSimpleMsiQuery (hDatabase, TEXT("INSERT INTO `_Validation` (`Table`, `Column`, `Nullable`, `MinValue`, `MaxValue`, `KeyTable`, `KeyColumn`, `Category`, `Set`, `Description`) VALUES (")
                                                    TEXT("'_Validation', 'KeyColumn', 'Y', 1, 32, NULL, NULL, NULL, NULL, 'Column to which foreign key connects')"));
        uStatus = ExecuteSimpleMsiQuery (hDatabase, TEXT("INSERT INTO `_Validation` (`Table`, `Column`, `Nullable`, `MinValue`, `MaxValue`, `KeyTable`, `KeyColumn`, `Category`, `Set`, `Description`) VALUES (")
                                                    TEXT("'_Validation', 'KeyTable', 'Y', NULL, NULL, NULL, NULL, 'Identifier', NULL, 'For foreign key, Name of table to which data must link')"));
        uStatus = ExecuteSimpleMsiQuery (hDatabase, TEXT("INSERT INTO `_Validation` (`Table`, `Column`, `Nullable`, `MinValue`, `MaxValue`, `KeyTable`, `KeyColumn`, `Category`, `Set`, `Description`) VALUES (")
                                                    TEXT("'_Validation', 'MaxValue', 'Y', -2147483647, 2147483647, NULL, NULL, NULL, NULL, 'Maximum value allowed')"));
        uStatus = ExecuteSimpleMsiQuery (hDatabase, TEXT("INSERT INTO `_Validation` (`Table`, `Column`, `Nullable`, `MinValue`, `MaxValue`, `KeyTable`, `KeyColumn`, `Category`, `Set`, `Description`) VALUES (")
                                                    TEXT("'_Validation', 'MinValue', 'Y', -2147483647, 2147483647, NULL, NULL, NULL, NULL, 'Minimum value allowed')"));
        uStatus = ExecuteSimpleMsiQuery (hDatabase, TEXT("INSERT INTO `_Validation` (`Table`, `Column`, `Nullable`, `MinValue`, `MaxValue`, `KeyTable`, `KeyColumn`, `Category`, `Set`, `Description`) VALUES (")
                                                    TEXT("'_Validation', 'Nullable', 'N', NULL, NULL, NULL, NULL, NULL, 'Y;N;@', 'Whether the column is nullable')"));
        uStatus = ExecuteSimpleMsiQuery (hDatabase, TEXT("INSERT INTO `_Validation` (`Table`, `Column`, `Nullable`, `MinValue`, `MaxValue`, `KeyTable`, `KeyColumn`, `Category`, `Set`, `Description`) VALUES (")
                                                    TEXT("'_Validation', 'Set', 'Y', NULL, NULL, NULL, NULL, 'Text', NULL, 'Set of values that are permitted')"));
        uStatus = ExecuteSimpleMsiQuery (hDatabase, TEXT("INSERT INTO `_Validation` (`Table`, `Column`, `Nullable`, `MinValue`, `MaxValue`, `KeyTable`, `KeyColumn`, `Category`, `Set`, `Description`) VALUES (")
                                                    TEXT("'_Validation', 'Table', 'N', NULL, NULL, NULL, NULL, 'Identifier', NULL, 'Name of table')"));
        uStatus = ExecuteSimpleMsiQuery (hDatabase, TEXT("INSERT INTO `_Validation` (`Table`, `Column`, `Nullable`, `MinValue`, `MaxValue`, `KeyTable`, `KeyColumn`, `Category`, `Set`, `Description`) VALUES (")
                                                    TEXT("'Property', 'Property', 'N', NULL, NULL, NULL, NULL, 'Identifier', NULL, 'Name of property, uppercase if settable by launcher or loader.')"));
        uStatus = ExecuteSimpleMsiQuery (hDatabase, TEXT("INSERT INTO `_Validation` (`Table`, `Column`, `Nullable`, `MinValue`, `MaxValue`, `KeyTable`, `KeyColumn`, `Category`, `Set`, `Description`) VALUES (")
                                                    TEXT("'Property', 'Value', 'N', NULL, NULL, NULL, NULL, 'Text', NULL, 'String value for property.  Never null or empty.')"));

        uStatus = MsiDatabaseCommit (hDatabase);
        // now we have MSI file which has 4608 Bytes 2560 -> it is 2048 bytes larger as an empty MSI
    }
    __finally {
        if (hFile != INVALID_HANDLE_VALUE)
            CloseHandle (hFile);

        if (hView != (MSIHANDLE)0)
            uStatus = MsiCloseHandle (hView);

        if (hSummaryInfo != (MSIHANDLE)0)
            uStatus = MsiCloseHandle (hSummaryInfo);

        if (hDatabase != (MSIHANDLE)0)
            uStatus = MsiCloseHandle (hDatabase);
    }
}

I am currently writing an application which requires me to call GetWindowText on arbitrary windows and store that data to a file for later processing. Long story short, I noticed that my tool was failing on Battlefield 3, and I narrowed the problem down to the following character in its window title: http://www.fileformat.info/info/unicode/char/2122/index.htm

So I created a little test app which just does the following:

std::wcout << L"\u2122";

Low and behold that breaks output to the console window for the remainder of the program.

Why is the MSVC STL choking on this character (and I assume others) when APIs like MessageBoxW etc display it just fine?

How can I get those characters printed to my file?

Tested on both VC10 and VC11 under Windows 7 x64.

Sorry for the poorly constructed post, I'm tearing my hair out here.

Thanks.

EDIT:

Minimal test case

#include <fstream>
#include <iostream>

int main()
{
  {
    std::wofstream test_file("test.txt");
    test_file << L"\u2122";
  }

  std::wcout << L"\u2122";
}

Expected result: '™' character printed to console and file. Observed result: File is created but is empty. No output to console.

I have confirmed that the font I"m using for my console is capable of displaying the character in question, and the file is definitely empty (0 bytes in size).

EDIT:

Further debugging shows that the 'failbit' and 'badbit' are set in the stream(s).

EDIT:

I have also tried using Boost.Locale and I am having the same issue even with the new locale imbued globally and explicitly to all standard streams.

To write into a file, you have to set the locale correctly, for example if you want to write them as UTF-8 characters, you have to add

const std::locale utf8_locale
            = std::locale(std::locale(), new std::codecvt_utf8<wchar_t>());
test_file.imbue(utf8_locale);

You have to add these 2 include files

#include <codecvt>
#include <locale>

To write to the console you have to set the console in the correct mode (this is windows specific) by adding

_setmode(_fileno(stdout), _O_U8TEXT);

(in case you want to use UTF-8).

For this you have to add these 2 include files:

#include <fcntl.h>
#include <io.h>

Furthermore you have to make sure that your are using a font that supports Unicode (such as for example Lucida Console). You can change the font in the properties of your console window.

The complete program now looks like this:

#include <fstream>
#include <iostream>
#include <codecvt>
#include <locale>
#include <fcntl.h>
#include <io.h>

int main()
{

  const std::locale utf8_locale = std::locale(std::locale(),
                                    new std::codecvt_utf8<wchar_t>());
  {
    std::wofstream test_file("c:\\temp\\test.txt");
    test_file.imbue(utf8_locale);
    test_file << L"\u2122";
  }

  _setmode(_fileno(stdout), _O_U8TEXT);
  std::wcout << L"\u2122";
}

I have a form1.cs and in that form I have a panel1, in the load event of the form1.cs I am adding a control to the panel1. Now my issue is, I have a control called Numbers.cs, I need to add another control to that panel1 but from this control in a button event. How can I do this?

public partial class Number : UserControl
{
    public Number()
    {
        InitializeComponent();
    }

    private void btnAcceptWelcome_Click(object sender, EventArgs e)
    {
          //HERE I NEED TO PASS A CONTROL TO THE PANEL1 IN FORM1.CS
          //NOT SURE HOW TO DO THIS.
    }
}

MORE INFO

So basically I have a folder called UserControls and in that folder I have

Numbers.cs
Letters.cs
Welcome.cs

All of them user controls, then i have a form

Form1.cs

Form1.cs instantiates Welcome and it is added to a Panel1 on the Form1.cs on form load. Then Welcome.cs has a button, when I click this button I need to swap to Numbers.cs. But I dont know how to do this from Welcome.cs

Another way would be to use a Custom Event raised by Numbers and handled by Form1 to pass the control and add it to your Panel's Control Collection.

---

This is an example of an Custom Event added to UserControl1

Form1

public partial class Form1 : Form
{
    UserControl2 mySecondControl = new UserControl2();
    public Form1()
    {
        InitializeComponent();
        userControl11.AddControl+=new EventHandler(SwapControls);

    }

    private void SwapControls(object sender, EventArgs e)
    {
        panel1.Controls.Remove(userControl11);
        userControl11.AddControl -= new EventHandler(SwapControls);
        panel1.Controls.Add(mySecondControl);
    }
}

UserControl

public partial class UserControl1 : UserControl
{
    public event EventHandler AddControl;
    public UserControl1()
    {
        InitializeComponent();
    }

    private void button1_Click(object sender, EventArgs e)
    {
        this.AddControl(this, new EventArgs());
    }
}