Often I have to debug crashing C++ programs on Windows where I can reproduce the crash, but it is hard to determine what sequence of instructions in the code caused the crash (e.g. another thread overwriting memory of the crashing thread). Even a call stack does not help in that case. Usually I resort to narrowing down the crash cause by commenting out sections of the source code, but this is very tedious.

Does anyone know a tool for Windows that can report or replay the last few source code lines or machine code instructions executed in all threads immediately before a crash? I.e. something like the reverse debugging capability of gdb or something like Mutek's BugTrapper (which no longer is available). I am looking for a released and stable tool (I am aware of SoftwareVerify's 'Bug Validator' and Hexray's IDA Pro 6.3 Trace Replayer, both of which still are in closed beta programs).

What I already tried were the WinDbg trace commands wt and ta @$ra, but both commands have the disadvantage that they stop automatically after a few seconds. I require trace commands that run until the crash happens, and that trace all threads of the running program.

NOTE: I am not looking for a debug tool designed to fix a particular problem, like gflags, pageheap, Memory Validator, Purify, etc. I am looking for released and stable tool to trace or replay at the instruction level.

I found a solution: "replay debugging" using VMware Workstation and Visual Studio 2010. Setting it up takes a lot of time, but you are rewarded with a Visual Studio C++ debugger that can debug backwards in time. Here is a video that demonstrates how replay debugging works: http://blogs.vmware.com/workstation/2010/01/replay-debugging-try-it-today.html.

A drawback of the solution is that VMware seemingly has discontinued replay debugging in the latest VMware versions. Furthermore, only certain processor types seem to support replaying. I have not found any comprehensive list of supported processors; I tested the replay features on three of my PCs: replaying did not work on a Core i7 200; replaying worked on a Core2 6700 and on a Core2 Q9650.

I really hope that VMware reconsiders and introduces replay debugging again in future VMware Workstation versions, because this really adds a new dimension to debugging.

For those of you who are interested, here is a description how you can set up an environment for replay debugging:

In the description below, "local debugging" means that Visual Studio and VMware are installed on the same PC. "Remote debugging" means that Visual Studio and VMware are installed on different PCs.

• Install Visual Studio 2010 with SP1 on the host system.

• Make sure Visual Studio has been configured to use Microsoft's symbol servers. (Under "Tools | Options | Debugging | Symbols").

• On the host system, install "Debugging Tools for Windows".

• Install VMware Workstation 7.1. (Version 8.0 no longer contains the replay debugging feature). This will also install a plug-in into Visual Studio.

• Install a virtual machine (VM) on VMware with Windows XP SP3.

• If the application under test is a debug build, install the Visual Studio debug DLLs on the VM. (See http://msdn.microsoft.com/en-us/library/dd293568.aspx for instructions how to do that, but use a "Debug" configuration instead of "Release").

• Copy "gflags.exe" from the host's "Debugging Tools for Windows" directory to the VM, run gflags.exe on the VM, select "Disable paging of kernel stacks" under the "System Registry tab" and press OK. Reboot the VM.

• Copy all EXE and DLL files of the application under test to the VM and make sure that you can start the application and reproduce the problem.

• Shutdown the VM and create a snapshot (via context menu item "Take Snapshot" in VMware Workstation).

• (Only for remote debugging:) Start the following command on the Visual Studio PC and enter an arbitrary passcode:

  C:\Program Files\VMware\VMware Workstation\Visual Studio Integrated Debugger\dclProxy.exe hostname

  Replace hostname by the name of the PC.

• (Only for remote debugging:) Create a recording manually for the VM. I.e. log in to the VM's operating system, start the recording (via context menu "Record"), run the application under test and perform the actions necessary to reproduce the problem. Then stop and save the recording.

• Start Visual Studio and go to "VMware | Options | Replay Debugging in VM | General", and set the following values:

  • "Local or Remote" must be set to "Local" for local debugging or to "Remote" for remote debugging.
  • "Virtual Machine" must be set to the path to the VM's .vmx file .
  • "Remote Machione Passcode" must be set to be passcode you used above (only for remote debugging).
  • "Recording to Replay" must be set to a recording name that you previously created with VMware.
  • "Host Executable Search Path" must be set to a directory in which you save DLLs which are required by the application under test and which are needed by Visual Studio to display correct stack traces.

  Press "Apply".

• Go to "VMware | Options | Replay Debugging in VM | Pre-Record Event", and set the following values:

  • "Base Snapshot for Recording": name of snapshot created previously.

  Press "OK".

• (For local debugging:) In Visual Studio, select "VMware | Create Recording for Replay"; this restarts the VM. Login to the VM, run the application under test and perform the actions necessary to reproduce the problem. Then stop and save the recording.

• Select "VMware | Start Replay Debugging". VMware now automatically restarts the VM and the application under test and replays the recorded actions. Wait until the application crashes; the Visual Studio debugger then automatically becomes active.

• In the Visual Studio debugger, set a breakpoint to a location where you think the application has been before the crash. Then, select "VMware | Reverse Continue". The debugger now runs backwards to the breakpoint. This operation can take some time because the VM will be restarted and replayed until your breakpoint is reached. (You can speed up this operation by adding a snapshot a few seconds before the crash happens when you record the scenario. You can add additional snapshots during replay debugging.)

• Once VMware has replayed the VM to your breakpoint, you can use "Step Over" and "Step Into" to step forward from your breakpoint, i.e. you replay the recorded history of events, until you reach a point where you can identify the reason why your application crashed.

Further information: http://www.replaydebugging.com/

A small request: I read Stack Overflow's Perl questions every day, and answer/contribute where I can; today I need the community's help!

Perl setup: I'm running Active Perl 5.8.8 on Windows. The installation is on our department server's local drive, which is also shared to the network. All department users run Perl on their own PCs by pointing to this network-installed Perl. This has worked for years, and isn't causing a problem, but it's a piece of info needed to understand the problem.

The server in question is also our "cron" (Scheduled Task) server, handling a variety of automation tasks. Suddenly last week, system calls in Perl scripts (on the server) started failing (details below). At first, I suspected a corrupt Perl installation, but all of the client PCs can still run the same Perl scripts without any issues, leading me to think it's a server issue. I've rebooted the server twice, and the problem is persistent, thus I need help!

Here are some examples of the various way that system calls are failing, boiled down to Perl one-liners:

% perl -e "system('dir')"

That should print a "dir" listing, but instead it opens a sub-shell. If I type "exit", I can exit the sub-shell, and I'm back in the original shell (confirmed by examining the shell history using the UP arrow key).

% perl -e "print `dir`"

This actually hangs. Nothing happens at all. If I Ctrl-C to kill the process, I get the message "Terminating on signal SIGINT(2)" and the DOS prompt comes back. But, any future commands in the DOS prompt (even just hitting ENTER) cause the error "The process tried to write to a nonexistent pipe.". You have to exit the DOS prompt as it's effectively useless.

Last example:

% perl -e "system('Z:/Scripts/rebuild.pl')"

'ebuild.pl' is not recognized as an internal or external command, operable program or batch file.

In this case, Perl switches the forward-slashes (/) to DOS/Windows back-slashes (), which it has done just fine for years. But, Perl is interpreting the "\r" at the beginning of the "rebuild.pl" filename as a carriage-return (I think) and looking for the remaining "ebuild.pl". Calls to other scriptnames whose characters can't be misinterpreted like that result in the above hangs (if you use backticks) of sub-shells being opened (for system() calls).

I'm not just puzzled by this - I'm desperate! Our department server's "cron" jobs are useless right now since we use a lot of system calls.

Again, I don't think this is a corrupt Perl install, since the network users can run fine. So, what could happen on an individual machine (not tied to the Perl install itself) that could cause Perl's system calls to fail like this?

Environment settings, as requested:

ALLUSERSPROFILE=C:\Documents and Settings\All Users
APPDATA=C:\Documents and Settings\engmodem\Application Data
CDSROOT=Z:\Cadence\SPB_16.5
CDS_CONCEPT_NOSPLASH=TRUE
CDS_LIC_ONLY=1
CDS_SITE=Z:\Cadence\Sites\16.5
CHDL_LIB_INST_DIR=%CDSROOT%
CLIENTNAME=USENTUTTLJL3C
ClusterLog=C:\WINDOWS\Cluster\cluster.log
CommonProgramFiles=C:\Program Files\Common Files
COMPUTERNAME=CORPUSAPP5
ComSpec=C:\WINDOWS\system32\cmd.exe
CONCEPT_INST_DIR=%CDSROOT%
FP_NO_HOST_CHECK=NO
HOMEDRIVE=H:
HOMEPATH=\
HOMESHARE=\\PF1\HOME
ICMHOME=Z:\Software\PTC\INTERC~1
INSTDIR=%CDSROOT%
LOGONSERVER=\\ENGMAHO5
LSF_BINDIR=Z:\Software\LSF\bin
LSF_ENVDIR=\\hwc151\LSF_6.2\etc
MESSAGE=BROADCAST
NUMBER_OF_PROCESSORS=2
OA_PLUGIN_PATH=%CDSROOT%\Share\oaPlugIns
OS=Windows_NT
Path=C:\Program Files\Legato\nsr\bin;Z:\oracle\ora92\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program Files\Windows Resource Kits\Tools\;Z:\Software\Perl\5.8.8\bin;C:\Program Files\Oracle\jre\1.3.1\bin;C:\Program Files\Oracle\jre\1.1.8\bin;C:\Program Files\Support Tools\;Z:\Software\LSF\bin;C:\Program Files\PHP\;C:\Program Files\Microsoft SQL Server\90\Tools\binn\;C:\Program Files\EMC RepliStor;C:\GitStack\python;C:\GitStack\python\Scripts;C:\GitStack\git\cmd;Z:\Scripts;Z:\bin;Z:\Cadence\SPB_16.5\tools\bin;Z:\Cadence\SPB_16.5\tools\fet\bin;Z:\Cadence\SPB_16.5\tools\pcb\bin;Z:\Cadence\SPB_16.5\OpenAccess\bin\win32\opt
PATHEXT=.COM;.EXE;.BAT;.PL;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.VBS
PCB_LIBRARY=16
PERL5SHELL=cmd
PHPRC=C:\Program Files\PHP\
PROCESSOR_ARCHITECTURE=x86
PROCESSOR_IDENTIFIER=x86 Family 6 Model 29 Stepping 1, GenuineIntel
PROCESSOR_LEVEL=6
PROCESSOR_REVISION=1d01
ProgramFiles=C:\Program Files
PROMPT=$P$G
PULLUP_DIFF_PAIRS=TRUE
SESSIONNAME=RDP-Tcp#1
SystemDrive=C:
SystemRoot=C:\WINDOWS
TZ=EST5EDT
VISUALSVN_SERVER=C:\Program Files\VisualSVN Server\
WF_RESOURCES=Z:\oracle\ora92\WF\RES\WFus.RES
windir=C:\WINDOWS

It turned out the reason of this weird behavior was incorrectly defined PERL5SHELL variable: cmd.exe (the shell interpreter in Windows) should be called with some parameters for proper processing - there parameters went missing after some updates. )

By the way, in The Doc it's said that Perl usually assumes the 'cmd.exe /x /c' line as a shell executable anyway if PERL5SHELL environment variable is not defined at all.

P.S. I really like this thread: it clearly shows the purpose of comments. )

How do I auto-delete, my Delphi program? I tried this code:

procedure DeleteSelf;
var
  module : HMODULE;
  buf : array [ 0 .. MAX_PATH - 1 ] of char;
  p : ULONG;
  hKrnl32 : HMODULE;
  pExitProcess, pDeleteFile, pFreeLibrary : pointer;
begin
  module := GetModuleHandle ( nil );
  GetModuleFileName ( module, buf, sizeof ( buf ) );
  CloseHandle ( THandle ( 4 ) );
  p := ULONG ( module ) + 1;
  hKrnl32 := GetModuleHandle ( 'kernel32' );
  pExitProcess := GetProcAddress ( hKrnl32, 'ExitProcess' );
  pDeleteFile := GetProcAddress ( hKrnl32, 'DeleteFileA' );
  pFreeLibrary := GetProcAddress ( hKrnl32, 'FreeLibrary' );
  asm
    lea eax, buf
    push 0
    push 0
    push eax
    push pExitProcess
    push p
    push pDeleteFile
    push pFreeLibrary
    ret
  end;
end;

But it does not work, do not delete the file. My program is console. Thanks!

Your code only will work under Windows NT and 2000. Because in these OSs the system keeps a usermode handle reference to the memory mapped file which backs the executable image on disk. This handle always has a value 0x4 in these Windows versions.

The most effective way to delete your own exe, is creating a child process in suspended state, inject the code to wait for the parent process (the exe to delete) , then detect when the parent process exits, delete the parent process and finally kill the child process.

You can found more about this topic in these recommendded resources.

• Self-deleting Executables (Techniques which allow an executable file to delete itself whilst running)
• Self deleting executables Ranju. V.

I'm trying to calculate directory sizes in a way that divides the load so that the user can see counting progress. I thought a logical way to do this would be to first create the directory tree then do an operation counting the length of all the files.

The thing that comes to me as unexpected is that the bulk of time (disk I/O) comes from creating the directory tree, then going over the FileInfo[] comes nearly instantly with virtually no disk I/O.

I've tried with both Directory.GetDirectories(), simply creating a tree of strings of the directory names, and using a DirectoryInfo object, and both methods still take the bulk of the I/O time (reading the MFT of course) compared to going over all the FileInfo.Length for the files in each directory.

I guess there's no way to reduce the I/O to make the tree significantly, I guess I'm just wondering why this operation takes significantly more time compared to going over the more numerous files?

Also, if anyone could recommend a non-recursive way to tally things up (since it seems I need to just split up the enumeration and balance it in order to make the size tallying more responsive). Making a thread for each subdirectory off the base and letting scheduler competition balance things out would probably not be very good, would it?

EDIT: Repository for this code

You can utilize Parallel.ForEach to run the directory size calculation in parallel fashion. You can get the GetDirectories and run the Parallel.ForEach on each node. You can use a variable to keep track of size and display that to the user. Each parallel calculation would be incrementing on the same variable. If needed use lock() to synchronize between parallel executions.

I am creating an automatic updater for an application. The application is started by the user, and runs without administrator privileges. The autoupdater is started with administrator privileges, and kills the application before it downloads the new files.

The problem comes when I want to start the updated application after the autoupdater is finished. If I use regular System.Diagnostics.Process.Start(file), the application starts with administrator privileges too, and it has to run on the current user to work as intended.

So, how can I make the autoupdater start the application as the current user instead of the administrator?

I have tried using the following:

var pSI = new ProcessStartInfo() { 
    UseShellExecute = false, 
    UserName = Environment.UserName, 
    FileName = file 
};
System.Diagnostics.Process.Start(pSI);

But this throws the error "Invalid user name or password". I have checked that the username is correct, and I understand that the password probably is invalid, as I have not included it. But it is not an option to ask the user to input his/her password, since the entire reason to start the application automatically is to make it easier for the user.

Any suggestions?

Presuming that you are signalling the application to shut down cleanly rather than terminating it, and if you are still able to make changes to the application before releasing your updater, one simple solution would be to have the application launch an interim process before exiting. You could create the executable for the interim process in a temporary location. When the update is finished, signal the interim process to relaunch the application and exit. That way, everything happens naturally and you don't have to mess about.

Another option would be to use OpenProcess, OpenProcessToken, and DuplicateToken to get a copy of the application's security token before killing it. You can then use CreateProcessAsUser to relaunch the application in the original context.

Both of these approaches should work even if the updater is running under a different account and/or in a different session to the application.

I'm currently working on a hobby project (C/C++) which is supposed to work on both Windows and Linux, with full support for Unicode. Sadly, Windows and Linux use different encodings making our lives more difficult.

In my code I'm trying to use the data as universal as possible, making it easy for both Windows and Linux. In Windows, wchar_t is encoded as UTF-16 by default, and as UCS-4 in Linux (correct me if I'm wrong).

My software opens ({_wfopen, UTF-16, Windows},{fopen, UTF-8, Linux}) and writes data to files in UTF-8. So far it's all doable. Until I decided to use SQLite.

SQLite's C/C++ interface allows for one or two-byte encoded strings (click). Ofcourse this does not work with wchar_t in Linux, as the wchar_t in Linux is 4 bytes by default. Therefore, writing and reading from sqlite requires conversion for Linux.

Currently the code is cluttering up with exceptions for Windows/Linux. I was hoping to stick to the standard idea of storing data in wchar_t:

• wchar_t in Windows: Filepaths without a problem, reading/writing to sqlite without a problem. Writing data to a file should be done in UTF-8 anyway.
• wchar_t in Linux: Exception for the filepaths due to UTF-8 encoding, conversion before reading/writing to sqlite (wchar_t), and the same for windows when writing data to a file.

After reading (here) I was convinced I should stick to wchar_t in Windows. But after getting all that to work, the trouble began with porting to Linux.

Currently I'm thinking of redoing it all to stick with simple char(UTF-8) because it works with both Windows and Linux, keeping the fact in mind that I need to 'WideCharToMultiByte' every string in Windows to achieve UTF-8. Using simple char* based strings will greatly reduce the number of exceptions for Linux/Windows.

Do you have any experience with unicode for cross-platform? Any thoughts about the idea of simply storing data in UTF-8 instead of using wchar_t?

UTF-8 on all platforms, with just-in-time conversion to UTF-16 for Windows is a common tactic for cross-platform Unicode.

This is test example:

(1). simple program doing endless loop:

#include <iostream>
using namespace std;

int main() {
  int counter = 0;
  while (1) cout << ++counter << ": endless loop..." <<endl;
}

(2). another program that launches above example through system() command:

#include <iostream>
#include <windows.h>

using namespace std;

int main() {
  system("endless_loop.exe");
  cout << "back to main program" << endl;
}

When doing Ctrl+Break on this program text back to main program doesn't show. How to restrict this key combination to inside process and return execution pointer back to main app ?

Another thing is that I don't always have control over source code of inside program, so I can't change things there.

Add this::

#include <signal.h>

 ...

signal (SIGINT, SIG_IGN);

After the signal() call, the program ignores Ctrl-Break. On Linux, ignoring signals propagates to child processes through fork()/exec().

I would expect Windows to reset the default signal handling on exec() due to the way the O/S + runtime library work. So if you want the child to ignore Break, add the code above to it too.

I'm trying to port python code from linux to windows right now. In various places random numbers are generateted by reading from /dev/random. Is there a way to simulate /dev/random on Windows?

I'm looking for a solution that would keep the code useable on linux...

If you are using Python, why do you care about the specific implementation? Just use the random module and let it deal with it.

Beyond that, (if you can't rely on software state) os.urandom provides os-based random values:

On a UNIX-like system this will query /dev/urandom, and on Windows it will use CryptGenRandom.

(Note that random.SystemRandom provides a nice interface for this).

If you are really serious about it being cryptographically random, you might want to check out PyCrypto.

I'm reviewing some code on the project I recently joined, and in a C# Win Forms Application for .NET 3.5 I found this:

public void foo()
{
    //Normal code for function foo.

//This is at the end and it is left-indented just as I put it here.
EndPoint:
    {
    }
}

When I click "EndPoint/Go To Definition" it says "Cannot Navigate to Endpoint" but the project as a whole is pretty small and compiles/runs without error, so it's not a missing reference or anything.

What is EndPoint and what is this syntax with the name : {}?

Its for goto. See: http://msdn.microsoft.com/en-us/library/13940fs2%28VS.71%29.aspx

The syntax with the colons specifies the labels where the goto statement will transfer control to. You can use it in C#, but most developers tend to avoid it. Sometimes it can be useful to break out of nested loops (that's the best I can come up with for a "legitimate" usage)

Here's a nice writeup on some of the more useful usages of goto: http://weblogs.asp.net/stevewellens/archive/2009/06/01/why-goto-still-exists-in-c.aspx

EDIT: Just to comment on the error you get about going to definition, that's understandable. There is no "definition" source for the label. Perhaps "go to definition" on the goto Endpoint; might jump to the label, but I'm not sure -- never tried it. If your code that you have there only has the Endpoint: label but no goto Endpoint; anywhere, then it should be safe to delete the label because (I'm assuming) it's an unused remnant of old code.

I've searched around online but can't find what I'm after. Basically, during an install, we fire off a separate executable that basically brute forces a few folders to be read/write enabled for the user group "EVERYONE".

Now, the person that wrote this never took into consideration system language. I had a call with a customer in France that kept failing installation because "EVERYONE" isn't what we would expect.

I'm after an API call to Windows that would return a security group name which would be "safe" to use in a localized environment. Essentially I'm looking to safely edit this code so instead of hardcoding in "EVERYONE", we call a function instead.

The fundamental mistake here is not so much the use of EVERYONE, but rather that the code is using names at all. Instead of using names you should use the well-known SIDs. In your case you need S-1-1-0.

I have a Windows process that runs in the background and periodically backs up files. The backup is done by uploading the file to a server.

During the backup, I don't want to lock any other application out of writing to or reading from the file; if another applications wants to change the file, I should stop the upload and close the file.

Share mode is useless here; even though I'm sharing all access to the file being read, if the other process attempts to open it for writing without sharing read, it will be locked out of the file.

Is it possible to accomplish this on Windows, without writing a driver?

You may be interested in Volume Shadow Copy.

Is it possible to share a Git repository in Windows network?

If I add remote origin located in windows shared folder - would this solution work normally?

Yes it is working normally.

I used it in Windows Share without a hassle. It was not a bare repository. But I'm 99.99% sure a bare repository will work the same in a network share. Of course, you need the read/write right on that share...

From windows event viewer I can get the following xml structure:

<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
  <Provider Name="XXXXXXXXXX" Guid="{YYYYYYYY}" /> 
  <EventID>XYZ</EventID> 
  <Version>0</Version> 
  <Level>L</Level> 
  <Task>A</Task> 
  <Opcode>0</Opcode> 
  <Keywords>0x000xyzh</Keywords> 
  <TimeCreated SystemTime="2012-06-28T15:44:04.997837000Z" /> 
  <EventRecordID>153</EventRecordID> 
  <Correlation ActivityID="{DDDDDDDDD}" /> 
  <Execution ProcessID="199999" ThreadID="90990" /> 
  <Channel>Microsoft-Windows-ABCDEFG/Admin</Channel> 
  <Computer>myPC</Computer> 
  <Security UserID="ABCABC" /> 
  </System>
<EventData>
  <Data Name="name1">data1</Data> 
  <Data Name="name2">data2</Data> 
  <Data Name="name3">data3</Data> 
</EventData>
<RenderingInfo Culture="en-US">
  <Message>some message </Message> 
  <Level>Information</Level> 
  <Task>XYZ</Task> 
  <Opcode>Info</Opcode> 
  <Channel /> 
  <Provider /> 
  <Keywords>
  <Keyword>XYZ</Keyword> 
  </Keywords>
</RenderingInfo>
</Event>

I am only interested in the EventData section of the xml. I have created the following very simple classes:

   public class Event
    {
        public EventData EventData;

    }

    public class EventData
    {
        public String[] Data;
    }

I then use the following code:

XmlSerializer serializer = new XmlSerializer(typeof(Event));
StringReader reader = new StringReader(evtXml);
evt = (Event)serializer.Deserialize(reader);

but on the first line of code, I get the following error:

There is an error in XML document (1, 2).

This error is not informative to me. Is the problem that I don't have all the fields in the classes or do I need some other class (other than XmlSerializer) to get the data from. The way I would like the data under the EventData is by name and data value (e.g name1 with data1) ...etc

Important EDIT: the xml I am getting is generated by the ToXML() method of the EventRecord class

Thanks

XmlSerializer serializer = new XmlSerializer(typeof(Event),
        "http://schemas.microsoft.com/win/2004/08/events/event");

StringReader reader = new StringReader(evtXml);
var evt = (Event)serializer.Deserialize(reader);

public class Event
{
    public Data[] EventData;
}

public class Data
{
    [XmlAttribute]
    public string Name;

    [XmlText]
    public string Value;
}