Vlad Ioan Topan

My playground

Archive for the ‘Snippets’ Category

Compiling a tiny executable (or DLL) in FASM (or MSVC) + templates – part II: MSVC

with one comment

Since creating tiny executables and DLLs in FASM was pretty straight-forward, we’ll challenge ourselves a bit: we’ll do it in MSVC. To be fully honest, creating a tiny executable using MSVC is actually more than a bit of a challenge, but it is doable.

An empty “int main(…) {return 0;}” program compiled with Microsoft’s Visual C++ compiler is already around 38 KB (using VS 2008, but other versions should yield similar results) due to the CRT (also calledcrt0, which implements (parts of) the C language runtime) code included in the executable. The CRT is responsible for things such as implementing the C API for working with files (fopen & co.), strings (strcpy & co.) etc., the malloc-based memory allocator, the well-known int main(…) entrypoint, etc.

[Sidenote] All Windows (PE) executables actually have a WinMain entrypoint, where a CRT stub is placed which, among many other things, parses the single-string command line and splits it into “parameters” passed as argv[] to main().

If we remove the CRT, we have to use Windows API functions to replace it (on Windows, obviously).

[Sidenote] We could alternatively link against a dynamic version of the runtime, but the code would still be much bigger than it needs to be, with the added pain of having a dependency on the Microsoft Visual C++ Redistributables, which have caused plenty of annoyance due to the somewhat cryptic error messages reported about some MSVCR##.DLL missing when the packages aren’t installed on the user’s machine.

The file management functions have good equivalents in the Windows API (start from CreateFile). The strings APIs are somewhat harder to find, but present. Memory allocation APIs are plentiful (LocalAlloc or HeapAlloc are good starts, or even VirtualAlloc for more complicated things). The fact that the WinMain() standard entrypoint does not provide “digested” arguments (like C’s argv/argc main() arguments) can also be handled using Windows API calls, but for some reason only the WideChar variant of CommandLineToArgv (i.e. CommandLineToArgvW) is implemented, so we’ll work with WideChar functions in the example.

Let’s create a 64-bit basic DLL loader and a sample DLL. First, the loader’s source code:

#include <windows.h>

int 
CALLBACK 
WinMain(
    _In_ HINSTANCE hInstance,
    _In_ HINSTANCE hPrevInstance,
    _In_ LPSTR lpCmdLine,
    _In_ int nCmdShow
    )
{
    WCHAR *cmdline;
    int argc;
    WCHAR **argv;
    
    /// we aren't interested in any of the standard WinMain parameters
    UNREFERENCED_PARAMETER(hInstance);
    UNREFERENCED_PARAMETER(hPrevInstance);
    UNREFERENCED_PARAMETER(lpCmdLine);
    UNREFERENCED_PARAMETER(nCmdShow);
    
    /// get the WideChar command line, split it
    cmdline = GetCommandLineW();
    argv = CommandLineToArgvW(cmdline, &argc);
   
    /// assume the first parameter is a DLL name, load it
    LoadLibraryW(argv[1]);

    /// free the "digested" parameters
    LocalFree(argv);
    
    ExitProcess(0);
}

In the interest of keeping things simple, this code is fugly and evil (it doesn’t check error codes), but it does work.

We have a standard WinMain entrypoint. We get the WideChar version of the command line using GetCommandLineW(), then
split it using CommandLineToArgvW() into the more familiar (to C devs at least) argc/argv pair. We call LoadLibraryW on
the first argument, which we assume is the DLL name, then free the argv obtained from CommandLineToArgvW() and exit.

The DLL is as basic as possible, just a header and a source (the header is given to serve as a template, it’s not actually necessary in this case).

The win64dll.h file:

#pragma once

#ifdef AWESOME_EXPORTS
#define AWESOME_API __declspec(dllexport)
#else
#define AWESOME_API __declspec(dllimport)
#endif

And the .c:

#include "win64dll.h"
#include <windows.h>

BOOL APIENTRY 
DllMain( 
    HMODULE hModule,
    DWORD  dwReason,
    LPVOID lpReserved
    )
{
    UNREFERENCED_PARAMETER(hModule);
    UNREFERENCED_PARAMETER(lpReserved);

    switch (dwReason)
    {
        case DLL_PROCESS_ATTACH:
            MessageBox(0, "Hello!", "Hello!", MB_OK);
            break;
        case DLL_THREAD_ATTACH:
        case DLL_THREAD_DETACH:
        case DLL_PROCESS_DETACH:
            break;
    }
    return TRUE;
}

Now for the hard part: compiling without the CRT. Since it’s easier to compile from the command line, or at least easier to write command line arguments than project settings/checkboxes in an article, let’s start off with compiling the loader above (presumably called loader.c) with the CRT and go from there:

cl.exe loader.c /link shell32.lib

Assuming your environment is set up correctly (see sidenote below about setting up the environment), this will produce a ~32 KB executable (on VS 2010 at least). The /link shell32.lib parameter is necessary because that lib contains the CommandLineToArgvW function we used. To get rid of the CRT, the /NODEFAULTLIB parameter is used. We’ll also need to explicitly define WinMain as the entrypoint, and add kernel32.lib to the library list, since it’s one of the “default” libs which is no longer automatically linked:

cl.exe loader.c /link shell32.lib kernel32.lib /nodefaultlib /entry:WinMain

[Sidenote] The linker parameters (the ones after /link) are NOT case-sensitive, either /nodefaultlib or /NODEFAULTLIB will work.

This should produce a 2560 byte executable, more than 12 times smaller than the CRT-based one, and doing exactly the same thing.

[Sidenote Q&A] Q. How to tell which .lib files are needed?
A. When the necessary .lib is not given as a command line argument, the compiler complains with an error message such as:

loader.obj : error LNK2019: unresolved external symbol __imp__LoadLibraryW@4 referenced in function _WinMain@16

This means the LoadLibraryW() function is missing it’s .lib; to find it, simply search for the LoadLibraryW text in the lib folder of the SDK (in a path similar to: “C:\Program Files (x86)\Microsoft SDKs\Windows\v7.1\Lib“) and use the lib it’s found in (in this case, kernel32.lib). As an alternative to searching in the (binary) .lib files, you can check the MSDN reference for the given function (search online for something like “MSDN LoadLibraryW”), which gives the lib filename in the “Library” section of the “Requirements” table at the bottom.

Compiling the DLL needs a few more paramters:

cl.exe win64dll.c /link shell32.lib kernel32.lib user32.lib /nodefaultlib /entry:DllMain /dll /out:win64dll.dll
  • user32.lib is needed for MessageBoxA
  • the /dll switch tells the linker to create a DLL
  • the exact name must be specified (via /out:win64dll.dll), because the linker defaults to creating .exe-s

This should yield a 3072 byte DLL. It’s slightly larger than the executable because of the export section (and the fact that each section’s size is rounded up to the FileAlignment (PE header) value, which defaults to 0x200 == 512, exactly the difference which we got, which also covers the small difference in size between the actual code produced).

After building the executable and the DLL, running:

loader.exe win64dll.dll

should pop up a “Hello!” message box, then exit.

[Sidenote] Setting up the cl.exe environment involves:

  • adding the VC\BIN subfolder of the MS Visual Studio installation to the %PATH% environment variable (should look like “C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin”)
  • adding to the %INCLUDE% environment variable the INCLUDE subfolders of:
    • the MS Platform SDK installation (e.g. “C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Include”)
    • the MS Visual Studio installation (e.g. “C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\Include”)
  • adding to the %LIB% environment variable the LIB subfolders of the VS and PSDK

The easiest way to do this is via a setupvc.bat file containing:

@echo off
SET VS_PATH=C:\Program Files (x86)\Microsoft Visual Studio 10.0
SET SDK_PATH=C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A
SET PATH=%VS_PATH%\VC\bin;%VS_PATH%\Common7\IDE;%PATH%
SET LIB=%VS_PATH%\VC\ATLMFC\LIB;%VS_PATH%\VC\LIB;%SDK_PATH%\LIB
SET INCLUDE=%VS_PATH%\VC\ATLMFC\INCLUDE;%VS_PATH%\VC\INCLUDE;%SDK_PATH%\include

After adjusting the paths as needed by your setup, run this before running cl.exe from the command line.

The first part of this text deals with tiny executables in FASM.

Advertisements

Written by vtopan

February 17, 2014 at 10:23 PM

Posted in C, programming, Snippets

Tagged with , , , , ,

Compiling a tiny executable (or DLL) in FASM (or MSVC) + templates – part I: FASM

with one comment

Having a tiny do-nothing executable can be useful for many purposes (or at least that’s what my malware-analysis-infested-background pushes me to believe). Nowadays, I use such executables to replace part of the half-a-dozen processes some evil software packages (*cough*Acrobat Reader*cough*) install on my system just to keep themselves updated. The same malware analysis experience kicks in again with an advice at this point: DON’T STOP SOFTWARE FROM UPDATING ITSELF unless you know what you’re doing!

Producing a tiny executable (32-bit PE to be precise) is easy with FASM (get FASM from here) and pretty straight-forward.

For example, save this as win32tiny.fasm:

format PE GUI 4.0
entry start

include 'win32a.inc'

;--------------------------------------------------------------------
section '.text' code readable executable

start:
    push 0              ;;; pass an ExitCode of 0 to ExitProcess
    call [ExitProcess]

;--------------------------------------------------------------------
section '.idata' import data readable writeable
library kernel, 'KERNEL32.DLL'
import kernel, ExitProcess, 'ExitProcess'

All this program does does is import the function ExitProcess from kernel32.dll and call it as soon as it is executed.
The “a” in win32a.inc stands for ASCII; we’ll go for the WideChar variant in the 64-bit executable later on.

To compile it, open up a command line (cmd.exe), add FASM’s \include subfolder to the %INCLUDE% environment variable and run FASM.EXE on the above source:

set INCLUDE=%INCLUDE%;c:\path_to_fasm\include
c:\path_to_fasm\FASM.EXE win32tiny.fasm

If all went well, you’ll end up with a win32tiny.exe (which does absolutely nothing) of only 1536 bytes.

[Sidenote] To make compiling sources with FASM easier, add a fasm.bat to your %PATH% environment variable with the following contents:

@echo off
set INCLUDE=%INCLUDE%;c:\path_to_fasm\include
c:\path_to_fasm\FASM.EXE %1

A 64-bit variant of the executable looks very similar:

format PE64 GUI 4.0
entry start

include 'win64a.inc'

;--------------------------------------------------------------------
section '.text' code readable executable

start:
xor rcx, rcx ;;; pass an ExitCode of 0 to ExitProcess
call [ExitProcess]

;--------------------------------------------------------------------
section '.idata' import data readable writeable
library kernel32,'KERNEL32.DLL'
import kernel32, ExitProcess,'ExitProcess'

The only differences are the PE64 format (which means the “PE32+” 64-bit format), the 64-bit version of the .inc file and passing the argument to ExitProcess via RCX, as per the Windows 64-bit calling convention (described here, with details about register usage here).

If we wanted to build a tiny DLL, things would actually be easier, since no imported functions are required (the DLL entrypoint simply returns “TRUE” to indicate successful loading):

format PE GUI 4.0 DLL
entry DllMain

include 'win32a.inc'

;;--------------------------------------------------------------------------------------------------
section '.text' code readable executable

proc DllMain hinstDLL, fdwReason, lpvReserved
mov eax, TRUE
ret
endp

The imports (“.idata”) section is gone, making the DLL only 1024 bytes long. The entrypoint changed to a DLL-specific one (see DllMain for details) and is now a proper function returning control to the loader (as opposed to the executables above, in which the call to ExitProcess makes any code after it irrelevant).

Building a 64-bit DLL simply requires adjusting the format to PE64 and the included header to the 64-bit variant, just like above.

Since we got this far, let’s have a look at a 64-bit DLL which might actually be used as a template, with proper imports and exports:

format PE64 GUI 4.0 DLL
entry DllMain</code>

include 'win64a.inc'

;--------------------------------------------------------------------
section '.text' code readable executable

proc Howdy
invoke MessageBox, 0, [title], NULL, MB_ICONERROR + MB_OK
ret
endp

proc DllMain hinstDLL, fdwReason, lpvReserved
mov eax, TRUE
ret
endp

;--------------------------------------------------------------------
section '.idata' import data readable writeable

library\
kernel, 'KERNEL32.DLL',\
user, 'USER32.DLL'

import kernel,\
GetLastError, 'GetLastError',\
FormatMessage, 'FormatMessageA',\
LocalFree, 'LocalFree'

import user,\
MessageBox, 'MessageBoxA'

;--------------------------------------------------------------------
section '.data' data readable writeable
title db "Hello, world!", 0

;--------------------------------------------------------------------
section '.edata' export data readable

export 'win64tiny.dll',\
Howdy, 'Howdy'

;--------------------------------------------------------------------
data fixups
end data

It has an exported function called “Howdy” which shows a message box, and a few more imports (some unused) to show how you can have more than one imported function per DLL. It also uses “invoke” to perform the call to MessageBox to keep things simple. The “data fixups” at the end generates relocation information, without which any real-life DLL would be unusable.

Part two of this text deals with doing this in MSVC.

Written by vtopan

February 16, 2014 at 7:57 PM

Funky Python – code snippets

with 3 comments

Python is a great programming language for a number of reasons, but one of it’s best features is the fact that, as per Python Zen item #13, “There should be one– and preferably only one –obvious way to do it.” Working with the beast for a number of years, however, does expose one to some less pythonic and somewhat quirky design points (maybe even a few gotchas); here are some of them.

Python quirks & gotchas

1. Boolean values in numeric contexts evaluate to 0/1

It’s very intuitive (especially when coming to Python from C) for 0 values to evaluate to False in boolean contexts and non-zero values to True. Having False evaluate to 0 and True to 1 in numeric contexts is however less intuitive (and somewhat useless):

>>> a = [1, 2, 3]
>>> a[True], a[False]
(2, 1)
>>> True + True # this one was rather unexpected...
2

2. The default argument values are evaluated at the point of function definition in the defining scope

This is probably one of the most frequent gotchas out there:

>>> def a(b=[]):
...     b.append(3)
...     print b
>>> a()
[3]
>>> a()
[3, 3]

The proper way to do this is to set b’s default value to None in the declaration and set it to [] inside the function body if it’s set to None:

>>> def a(b=None):
...     if b is None: b = []        
...     b.append(3)

3. *Everything* is an object

Although it’s a fact which may escape even the best of programmers at the beginning, Python is actually object oriented to the core. Despite the fact that it allows you to write procedural (and even functionalish) code, everything is an object. Functions are objects, data types are objects etc. This:

69.bit_length()

doesn’t work because the period is parsed as part of the numeric token (think of 69.j or 69.e1); this however:

(69).bit_length()

works. Being objects, functions can also have attributes:

>>> def a():
...    print a.x
>>> a.x = 3
>>> a()
3

This comes in handy e.g. for giving a decorated function the same internal name (for introspection purposes) as the original function:

def print_call_decorator(fun):
    def replacement(*args, **kwargs):
        res = fun(*args, **kwargs)
        print r'Call %s.%s => %s' % (inspect.getmodule(fun).__name__, fun.__name__, res)
        return res
    replacement.__name__ = fun.__name__
    return replacement

4. Generators, sets & dictionaries also have comprehensions (called “displays”)

As you probably know, list comprehensions are a great way to generate a new list from another iterable. But it goes further… Generators, sets and dicts also have something similar, called displays. The basic syntax is this:
generator = (value for ... in ...)
dict = {key:value for ... in ...}
set = {value for ... in ...}

E.g.:

>>> a = ['a', 'b', 'c']
>>> d = {x:a.index(x) for x in a}
>>> d
{'a': 0, 'c': 2, 'b': 1}
>>> d_rev = {d[x]:x for x in d}
>>> d_rev
{0: 'a', 1: 'b', 2: 'c'}

This makes reversing a dictionary for example much cleaner.
What makes displays even more fun are the *list* displays, which are essentially list comprehensions but with unlimited depth; using them to flatten a list of lists would look something like this:

flat = [x for y in lst for x in y]

The x/y order in the example is not a mistake; that’s actually the proper order.

5. GIL: Python threads aren’t

Not on multi-processor machines, at least. Yes, there is a threading module (aptly named), but due to the Global Interpreter Lock, threads of the same (Python) process can’t actually run at the same time. This becomes more of an issue when deploying native-Python servers, as they don’t get any benefit from the number of cores installed on the machine (drastically limiting the number of open sockets a Python process can handle at the same time as opposed to a native one written in C).

6. for has an else clause

…and so do the try…except/finally and while constructs. In all cases, the else branch is executed if all went well (break wasn’t called to stop cycles / no exception occurred). And while the else branch may be useful to perform when you want something to happen only if the cycle construct wasn’t “broken” (the classic example is handling the fact that the cycle hasn’t found the value it was looking for), try doesn’t really need an else clause, as the following are equivalent and the latter seems at least to me more readable:

  • with else:
    try:
        this_may_crash()
    except:
        handle_it()
    else:
        call_me_if_it_didnt_crash()
  • without else:
    try:
        this_may_crash()
        call_me_if_it_didnt_crash()
    except:
        handle_it()

7. Tuple assignment order

In tuple assignments, the left-values are eval’ed & assigned in order:

>>> a = [1, 2, 3]
>>> i, a[i] = 1, 5 # i is set to 1 *before* a[i] is evaluated
>>> a
[1, 5, 3]

This happens because tuple assignments are equivalent to assigning the unpacked pairs in order; the second line above is therefore equivalent to:

>>> i = 1
>>> a[i] = 5

8. Scope juggling

Inside a function, variables are resolved in the global scope if no direct variable assignment appears in the
function, but are local otherwise (making Python clairvoyant, as it is able to tell that something is going to happen later on inside the function, i.e. a variable will be set). Note that attributes and sequence/dict values can still be set, just not the “whole” variable…

a1 = [1, 2, 3]
a2 = [1, 2, 3]
b = 3
c = 4
def fun():
    global b
    print c # crashes, because c is resolved to the local one (which is not set at this point)
    print b # works, because the global directive above forces b to be resolved to the global value
    a1[0] = 4 # works, because a1 is not directly set anywhere inside the function
    a2[0] = 5 # crashes, because a2 is later on directly set
    c = 10
    b = 11
    a2 = 'something else'

Bonus facts

As a bonus for making it through to the end, here are some lesser known / less frequently pondered upon facts about Python:

  1. a reference to the current list comprehension (from the inside) can be obtained with:
    locals()['_[1]'].__self__
  2. list comprehensions can contain any number of forin levels (this is actually documented). Flattening lists:
    flat = [x for y in lst for x in y]
  3. range() actually builds a list, which can be slow and memory-consuming for large values; use xrange()
  4. modules have a dict attribute .__dict__ with all global symbols
  5. the sys.path list can be tampered with before some imports to selectively import modules from dynamically-generated paths
  6. flushing file I/O must be followed by an os.fsync(…) to actually work:
    f.flush()
    os.fsync(f.fileno())
  7. after instantiation, object methods have the read-only attributes .im_self and .im_func set to the current object’s class and the implementing function respectively
  8. some_set.discard(x) removes x from the set only if present (without raising an exception otherwise)
  9. when computing actual indexes for sequences, negative indexes get added with the sequence length; if the result is still negative, IndexError is raised (so [1, 2, 3][-2] is 2 and [1, 2, 3][-4] raises IndexError)
  10. strings have the .center(width[, fillchar]) method, which padds them centered with fillchar (defaults to space) to the length given in width
  11. inequality tests can be chained: 1 < x < 2 works
  12. the minimum number of bits required to represent an integer (or long) can be obtained with the integer’s .bit_length() method

Written by vtopan

March 17, 2011 at 12:37 AM

Posted in Python, Snippets

Listing the modules (DLLs) of the current process without API functions

with one comment

There is a simple way to walk the list of loaded modules (DLLs) of the current Windows process without calling any API functions. It also works with other processes, but it obviously involves reading the respective process’ memory space, which in turn involves using the aptly-named ReadProcessMemory function, defeating the whole purpose of not using APIs.

The PEB and the TIB

The PEB (Process Enviroment Block), briefly documented here (the page which passes as “documentation” on MSDN just… isn’t) is a structure (stored for each process in it’s own memory space) which holds various process parameters used by the OS such as the PID, a flag set if the process is being debugged, some localization information etc. The PEB is pointed to by the TIB (Thread Information Block), which is always located at FS:[0] (if that sounds like black magic, either pick up a book on assembly language and/or x86 processor architecture, or ignore that piece of information and use the code below to get to it).

One of the PEB entries is a list of, well, the loaded modules of the process. Actually, a PEB member points to a structure called LoaderData (it’s type being PEB_LDR_DATA), again “almost documented” by MS, which points to the list we’re interested in, looking something like this:

typedef struct LDR_DATA_ENTRY {
  LIST_ENTRY              InMemoryOrderModuleList;
  PVOID                   BaseAddress;
  PVOID                   EntryPoint;
  ULONG                   SizeOfImage;
  UNICODE_STRING          FullDllName;
  UNICODE_STRING          BaseDllName;
  ULONG                   Flags;
  SHORT                   LoadCount;
  SHORT                   TlsIndex;
  LIST_ENTRY              HashTableEntry;
  ULONG                   TimeDateStamp;
  } LDR_DATA_ENTRY, *PLDR_DATA_ENTRY;

It’s a linked list (the LIST_ENTRY structure contains the forward pointer called Flink) which contains serveral useful pieces of information about each module, among which it’s image base (BaseAddress), virtual address of the entrypoint (EntryPoint) (NOT the RVA you find in the PE header, but the actual computed VA) and DLL name with (FullDllName) and without (BaseDllName) the path, as UNICODE_STRINGs. The list is circularly linked and has a sentinel element with the BaseAddress member set to 0.

The list entries should be of the LDR_DATA_TABLE_ENTRY data type described by MS here, but the actual structure found in-memory doesn’t have the first member (BYTE Reserved1[2]). An alternative (and more complete) definition of the structure can be found here, but it has three LIST_ENTRY members at the beginning instead of just one.

Getting a pointer to the list

We retrieve the pointer to the list from the PEB using inline assembly from the C source code:

__declspec(naked)
PLDR_DATA_ENTRY firstLdrDataEntry() {
   __asm {
      mov eax, fs:[0x30]  // PEB
      mov eax, [eax+0x0C] // PEB_LDR_DATA
      mov eax, [eax+0x1C] // InInitializationOrderModuleList
      retn
      }
   }

There are actually three lists of modules, sorted by three different criterias; the one we’re using is sorted by the order in which the modules were loaded.

Example

Using the structure defined above and the firstLdrDataEntry function, it becomes trivial to walk the list of loaded modules:

void main() {
   PLDR_DATA_ENTRY cursor;
   cursor = firstLdrDataEntry();
   while (cursor->BaseAddress) {
      printf( "Module [%S] loaded at [%p] with entrypoint at [%p]\n",
              cursor->BaseDllName.Buffer, cursor->BaseAddress,
              cursor->EntryPoint);
      cursor = (PLDR_DATA_ENTRY)cursor->InMemoryOrderModuleList.Flink;
      }
   }

Written by vtopan

May 27, 2009 at 11:22 PM

Using NtDeleteFile from Delphi

with 2 comments

The NtDeleteFile API

The native API NtDeleteFile performs the same task as the user-mode API DeleteFile, but interrestingly enough, the user-mode API does not call the native API to perform it’s task. As explained here, normally files are deleted through calls to NtSetInformationFile. The main difference in behavior comes from the fact that NtDeleteFile does not wait for handles on the file to close before deleting it (note that if the file is “open for normal I/O or as a memory-mapped file”, it still can’t be deleted, so only read-only handles will be ignored).

Data structures

The required structures (not defined in Delphi) are UNICODE_STRING and OBJECT_ATTRIBUTES.

Steps

The steps to remove the file are:

  • convert the “DOS” path name to an “NT” path name (basically prepend “\??\” to whatever the plain path is) using RtlDosPathNameToNtPathName_U;
  • fill in the OBJECT_ATTRIBUTES structure;
  • call NtDeleteFile.

Importing the APIs

Using the afore-linked-to type definitions, we still need to import the native APIs:

function NtDeleteFile(ObjectAttributes:POBJECT_ATTRIBUTES):DWORD; stdcall; external 'ntdll.dll';
function RtlDosPathNameToNtPathName_U(DosName:PWChar; var NtName:UNICODE_STRING; DosFilePath:PPChar; NtFilePath:PUNICODE_STRING):BOOL; stdcall; external 'ntdll.dll';

Do note that this statically links the imported functions, making the whole application unable to load if the undocumented APIs are not present on the system (at least for Windows 2000 and XP they should be).

Converting DOS paths to native paths using RtlDosPathNameToNtPathName_U

Converting the DOS path name to a native path is done by the RtlDosPathNameToNtPathName_U API, which takes in a PWChar argument containing the DOS path and a pre-allocated UNICODE_STRING structure of MAX_PATH WideChars and returns True if it has successfully converted the path.

Source code

The code looks something like this:

function _DeleteFile(fileName:string):DWORD;
var
   oa:OBJECT_ATTRIBUTES;
   ws1, ws2:WideString;
   us:UNICODE_STRING;
begin
   Result := $C0000001; // STATUS_UNSUCCESSFUL, "generic" error 
   ws1 := fileName; // automatic String -> WideString conversion
   SetLength(ws2, MAX_PATH);
   us.Length := MAX_PATH;
   us.MaximumLength := MAX_PATH;
   us.Buffer := @ws2[1];
   if not RtlDosPathNameToNtPathName_U(@ws1[1], us, nil, nil)
      then Exit;
   oa.Length := SizeOf(OBJECT_ATTRIBUTES);
   oa.RootDirectory := 0;
   oa.ObjectName := @us;
   oa.Attributes := $40; // case insensitive
   oa.SecurityDescriptor := nil;
   oa.SecurityQualityOfService := nil;
   Result := NtDeleteFile(@oa); // pass on the NTSTATUS
end;

Written by vtopan

May 26, 2009 at 3:43 PM

Posted in Delphi, Snippets

Tagged with , ,

How to check if a thread/process is suspended (get thread state)

with 4 comments

The basic steps to get to a thread’s status information is the following (knowing of course both the process ID (hence forth PID) and the thread ID (TID)):

  1. call NtQuerySystemInformation with SystemInformation set to SystemProcessInformation (5)
  2. iterate over the array of SYSTEM_PROCESS_INFORMATION structures (the structure contents is (wrongfully) explained here; correct version here) to find your PID (ProcessId member) of interest
  3. iterate over the array of SYSTEM_THREAD structures (detailed below) to find the desired TID (UniqueThread member) and check the State and WaitReason members; both must be set to 5 if the thread is suspended, any other values otherwise

As it’s probably obvious to most people keen on system-level programming, a process is suspended when all it’s threads are suspended, so all of them must be checked for the suspended status.

Step one: calling NtQuerySystemInformation

The required structures are defined here (for Delphi). The function isn’t defined in any headers, so we must declare it’s prototype ourselves:

function NtQuerySystemInformation(SystemInformationClass:DWORD; SystemInformation:pointer; SystemInformationLength:DWORD; ReturnLength:PDWORD):cardinal; stdcall; external 'ntdll';

Example usage:

var 
   spi:PSYSTEM_PROCESS_INFORMATION;
   size:DWORD;
begin
if (NtQuerySystemInformation(5, nil, 0, @size) = STATUS_INFO_LENGTH_MISMATCH) // SystemProcessInformation
   and (size > 0)
   then begin
        GetMem(spi, size);
        if NtQuerySystemInformation(5, spi, size, @size) = 0
           then begin
                [...] // do something with spi
                end
           else HandleError; // failed listing processes!
        FreeMem(spi);
        end
    else HandleError; // failed listing processes!
end;

HandleError is a fictional function (which you’ll most likely skip, ’cause you’re in a hurry to get things done, right? 🙂 ).

Step two: iterating the process list

The structure only looks like a linked list item; the NextEntryOffset member is an actual offset from the beginning of the current structure to the beginning of the next one. This is needed because of the variable size of the structure (given by the variable number of threads for each process). We need an extra crt:PSYSTEM_PROCESS_INFORMATION variable to walk the pseudo-linked list because we must keep the original psi pointer to free it’s memory.
The outline of the code which iterates the processes looking for a PID (given the spi:PSYSTEM_PROCESS_INFORMATION pointer from above) would look like this:

var 
    crt:PSYSTEM_PROCESS_INFORMATION; 
[...]
    crt := spi;
    repeat
        if crt^.ProcessID = PID
           then begin
                [...] // do something with crt^
                break;
                end;
        crt := Pointer(DWORD(crt) + crt^.NextEntryOffset);
    until crt^.NextEntryOffset = 0;

Step three: find the appropriate thread

Given the ThreadInfo array in the structure located at the previous step, we iterate through it and test the State and WaitReason members for the item matching our TID:

var
    j:integer;
[...]
    for j := 0 to crt^.NumberOfThreads-1 do
        begin
        if crt^.ThreadInfo[j].UniqueThread = TID
           then begin
                if crt^.ThreadInfo[j].WaitReason = 5
                   then [...] // the thread is suspended
                   else [...]; // the thread is not suspended
                break; 
                end;
        end;

The State member must also be set to 5 (“waiting”), but if the WaitReason is non-null, the State must be 5 (and vice-versa), so there’s little point in checking it explicitly.

Additional info: thread starting address, priority etc.

If you’ve paid any attention while reading the structures, you might have noticed additional interesting information about threads and processes, such as the creation time, image path, priority, handle count and memory and I/O usage/history for processes (this is how Process Explorer gets, for example, the WorkingSet/PeakWorkingSet and ReadBytes/WriteBytes/OtherBytes information) and starting address, priority/base priority and various timing information for threads. The starting address is particularly interesting, because the NtQueryInformationThread API with ThreadInformationClass set to ThreadQuerySetWin32StartAddress (9) only works (on Windows pre-Vista) “before the thread starts running” (quoted from MSDN), which seems to me rather pointless in the first place.

The NtQuerySystemInformation API is also a useful replacement for the CreateToolhelp32Snapshot suite, yielding more information about processes and threads.

Written by vtopan

April 15, 2009 at 1:07 AM

Posted in Delphi, Snippets

Python snippets (1)

leave a comment »

Apparently, Python 2.6 supports a max. of 100 named groups in regular expressions. It also doesn’t support constructs such as (?i:…), which makes it impossible to selectively mark groups as case insensitive in a regex with more than 100 groups. The performance penalty for making the complete regex case insensitive is enormous (roughly double running time), so the only solution to my problem (which involves searching for a large number of patterns, some of them case insensitive) appears to be linking to an external regex module, such as PCRE (which will require bindings for Python, as PCRE for example doesn’t have any (not any more, at least)), or implementing my own pattern searching library. This should be fun.

Written by vtopan

February 18, 2009 at 8:29 PM