Vlad Ioan Topan

The NtDeleteFile API

The native API NtDeleteFile performs the same task as the user-mode API DeleteFile, but interrestingly enough, the user-mode API does not call the native API to perform it’s task. As explained here, normally files are deleted through calls to NtSetInformationFile. The main difference in behavior comes from the fact that NtDeleteFile does not wait for handles on the file to close before deleting it (note that if the file is “open for normal I/O or as a memory-mapped file”, it still can’t be deleted, so only read-only handles will be ignored).

Data structures

The required structures (not defined in Delphi) are UNICODE_STRING and OBJECT_ATTRIBUTES.

Steps

The steps to remove the file are:

• convert the “DOS” path name to an “NT” path name (basically prepend “\??\” to whatever the plain path is) using RtlDosPathNameToNtPathName_U;
• fill in the OBJECT_ATTRIBUTES structure;
• call NtDeleteFile.

Importing the APIs

Using the afore-linked-to type definitions, we still need to import the native APIs:

function NtDeleteFile(ObjectAttributes:POBJECT_ATTRIBUTES):DWORD; stdcall; external 'ntdll.dll';
function RtlDosPathNameToNtPathName_U(DosName:PWChar; var NtName:UNICODE_STRING; DosFilePath:PPChar; NtFilePath:PUNICODE_STRING):BOOL; stdcall; external 'ntdll.dll';

Do note that this statically links the imported functions, making the whole application unable to load if the undocumented APIs are not present on the system (at least for Windows 2000 and XP they should be).

Converting DOS paths to native paths using RtlDosPathNameToNtPathName_U

Converting the DOS path name to a native path is done by the RtlDosPathNameToNtPathName_U API, which takes in a PWChar argument containing the DOS path and a pre-allocated UNICODE_STRING structure of MAX_PATH WideChars and returns True if it has successfully converted the path.

Source code

The code looks something like this:

function _DeleteFile(fileName:string):DWORD;
var
   oa:OBJECT_ATTRIBUTES;
   ws1, ws2:WideString;
   us:UNICODE_STRING;
begin
   Result := $C0000001; // STATUS_UNSUCCESSFUL, "generic" error 
   ws1 := fileName; // automatic String -> WideString conversion
   SetLength(ws2, MAX_PATH);
   us.Length := MAX_PATH;
   us.MaximumLength := MAX_PATH;
   us.Buffer := @ws2[1];
   if not RtlDosPathNameToNtPathName_U(@ws1[1], us, nil, nil)
      then Exit;
   oa.Length := SizeOf(OBJECT_ATTRIBUTES);
   oa.RootDirectory := 0;
   oa.ObjectName := @us;
   oa.Attributes := $40; // case insensitive
   oa.SecurityDescriptor := nil;
   oa.SecurityQualityOfService := nil;
   Result := NtDeleteFile(@oa); // pass on the NTSTATUS
end;

The basic steps to get to a thread’s status information is the following (knowing of course both the process ID (hence forth PID) and the thread ID (TID)):

1. call NtQuerySystemInformation with SystemInformation set to SystemProcessInformation (5)
2. iterate over the array of SYSTEM_PROCESS_INFORMATION structures (the structure contents is (wrongfully) explained here; correct version here) to find your PID (ProcessId member) of interest
3. iterate over the array of SYSTEM_THREAD structures (detailed below) to find the desired TID (UniqueThread member) and check the State and WaitReason members; both must be set to 5 if the thread is suspended, any other values otherwise

As it’s probably obvious to most people keen on system-level programming, a process is suspended when all it’s threads are suspended, so all of them must be checked for the suspended status.

Step one: calling NtQuerySystemInformation

The required structures are defined here (for Delphi). The function isn’t defined in any headers, so we must declare it’s prototype ourselves:

function NtQuerySystemInformation(SystemInformationClass:DWORD; SystemInformation:pointer; SystemInformationLength:DWORD; ReturnLength:PDWORD):cardinal; stdcall; external 'ntdll';

Example usage:

var 
   spi:PSYSTEM_PROCESS_INFORMATION;
   size:DWORD;
begin
if (NtQuerySystemInformation(5, nil, 0, @size) = STATUS_INFO_LENGTH_MISMATCH) // SystemProcessInformation
   and (size > 0)
   then begin
        GetMem(spi, size);
        if NtQuerySystemInformation(5, spi, size, @size) = 0
           then begin
                [...] // do something with spi
                end
           else HandleError; // failed listing processes!
        FreeMem(spi);
        end
    else HandleError; // failed listing processes!
end;

HandleError is a fictional function (which you’ll most likely skip, ’cause you’re in a hurry to get things done, right? ).

Step two: iterating the process list

The structure only looks like a linked list item; the NextEntryOffset member is an actual offset from the beginning of the current structure to the beginning of the next one. This is needed because of the variable size of the structure (given by the variable number of threads for each process). We need an extra crt:PSYSTEM_PROCESS_INFORMATION variable to walk the pseudo-linked list because we must keep the original psi pointer to free it’s memory.
The outline of the code which iterates the processes looking for a PID (given the spi:PSYSTEM_PROCESS_INFORMATION pointer from above) would look like this:

var 
    crt:PSYSTEM_PROCESS_INFORMATION; 
[...]
    crt := spi;
    repeat
        if crt^.ProcessID = PID
           then begin
                [...] // do something with crt^
                break;
                end;
        crt := Pointer(DWORD(crt) + crt^.NextEntryOffset);
    until crt^.NextEntryOffset = 0;

Step three: find the appropriate thread

Given the ThreadInfo array in the structure located at the previous step, we iterate through it and test the State and WaitReason members for the item matching our TID:

var
    j:integer;
[...]
    for j := 0 to crt^.NumberOfThreads-1 do
        begin
        if crt^.ThreadInfo[j].UniqueThread = TID
           then begin
                if crt^.ThreadInfo[j].WaitReason = 5
                   then [...] // the thread is suspended
                   else [...]; // the thread is not suspended
                break; 
                end;
        end;

The State member must also be set to 5 (“waiting”), but if the WaitReason is non-null, the State must be 5 (and vice-versa), so there’s little point in checking it explicitly.

Additional info: thread starting address, priority etc.

If you’ve paid any attention while reading the structures, you might have noticed additional interesting information about threads and processes, such as the creation time, image path, priority, handle count and memory and I/O usage/history for processes (this is how Process Explorer gets, for example, the WorkingSet/PeakWorkingSet and ReadBytes/WriteBytes/OtherBytes information) and starting address, priority/base priority and various timing information for threads. The starting address is particularly interesting, because the NtQueryInformationThread API with ThreadInformationClass set to ThreadQuerySetWin32StartAddress (9) only works (on Windows pre-Vista) “before the thread starts running” (quoted from MSDN), which seems to me rather pointless in the first place.

The NtQuerySystemInformation API is also a useful replacement for the CreateToolhelp32Snapshot suite, yielding more information about processes and threads.

If you’ve ever taken a shot at porting headers from C to Delphi, you may have noticed the different way in which the struct/record fields are aligned, but most likely you havent. And if you had the bad luck of stumbling over a struct that gets aligned differently when ported to Delphi, even though the data types were translated correctly, you’re probably interested in what happens behind the scenes and how to fix it. Well, here it goes.

Why variable addresses are aligned in memory

As explained in depth here, because of it’s internal structure, the time required by the CPU to access data from the RAM depends on the alignment of the address it requests (the “aligned” memory addresses being accessed faster). Aligned means that the address should be a multiple of the CPU’s memory word (not to be confused with the WORD data type in C/Delphi, which is always 2 bytes), which is the least amount of memory the CPU can get from RAM at one time. It’s also the size of the general purpose registers on the CPU, and it represents the “chunk” of data the CPU is most “comfortable” working with. On x86 architectures it’s 32 bits, which means 4 bytes (on 64 bit architectures it’s… well, 64 bits = 8 bytes). So, the memory addresses should be multiples of 4 on x86 systems for faster access.

Nitpicking corner: why is the WORD type 2 bytes long then, if the memory word on x86 is 4 bytes? Well, it’s not actually *all* the x86 architectures that have a 4 byte memory word, just the 386-and-higher ones (the aptly-named 32-bit architectures). But back when dinosaurs still roamed the Earth and “640K of memory were enough for anybody” (to paraphrase (?) one of the foremost computer visionaries), the computer word was just 16 bits, or 2 bytes, and that’s when the WORD data type was coined (without very much foresight, truth be told).

Struct field alignment in C

According to this paragraph on Wikipedia, struct member fields are aligned based on their size by padding them with, basically, their size in bytes. So, assuming we declare the structure:

typedef struct {
BYTE a;
WORD b;
BYTE c;
DWORD d;
} foo;

you might be tempted to assume it will take 8 bytes of memory. But, due to the afore-mentioned alignment, b will be aligned on a 2-byte boundary, and c on a 4-byte boundary, so that structure will actually be equivalent to this one:

typedef struct {
BYTE a;
BYTE padding_for_b[1]; // 1-byte padding to align .b
WORD b;
BYTE c;
BYTE padding_for_d[3]; // 3-byte padding to align .d
DWORD d;
} foo;

and take precisely 12 bytes. You can force C’s hand not to align the members using the #pragma pack(1) directive, but (a) the code will be slower and (b) you can’t do that to standard structures in the Windows API for obvious reasons.

Record field alignment in Delphi

As explained in the “align fields compiler directive” help entry, in Delphi the record field alignment is not based on the field’s data type size, but on a constant specified by the ALIGN (A) directive (which can also be set for the entire project from Project->Options->Compiler->Record field alignment.
The default is {$ALIGN 8} (short: {$A8}), which means alignment to an 8-byte boundary. However, that’s how things work only in theory. In practice, for some mysterious reason, Delphi does exactly what C does: unless the alignment is set to 1 (eg. by {$A1}), it aligns record fields based on their size.

The difference

But if Delphi behaves just like C, why do things sometimes still crash? They do because of one exception to the rule: enumerations are treated differently. In C, they are 4 bytes in size by default (this can be changed using the #pragma enum directive). In Delphi, the enumeration size is controlled by the $Z directive, and by default it’s set to 1. Therefore, in order to get the proper translation of API structures from C to Delphi, you should use the {$Z4} directive.

In the process of writing a Process Explorer meets HijackThis one-size-fits-all monster app, I faced the rather tedious task of porting Windows SDK headers (some less documented than others) to Delphi. As expected, I ran into annoying gotchas; here are some of them.

Boolean type in C / Windows SDK

The C specs didn’t define a boolean type up until C99 (which introduced the bool data type), which led to a variety of “standards”. The SDK uses the BOOLEAN type, which is actually a BYTE, and should be mapped to the Boolean type in Delphi, not BOOL/LongBool (yes, I am well aware that this may be a “duh!” for some).

Other type mappings

UCHAR is unsigned char, not “unicode char” as some might expect, so it translates to BYTE, not WORD.

Struct to record mappings

See this post.

[to be continued]