Vlad Ioan Topan

Funky Python – code snippets

vtopan — Thu, 17 Mar 2011 00:37:15 +0000

Python is a great programming language for a number of reasons, but one of it’s best features is the fact that, as per Python Zen item #13, “There should be one– and preferably only one –obvious way to do it.” Working with the beast for a number of years, however, does expose one to some less pythonic and somewhat quirky design points (maybe even a few gotchas); here are some of them.

Python quirks & gotchas

1. Boolean values in numeric contexts evaluate to 0/1

It’s very intuitive (especially when coming to Python from C) for 0 values to evaluate to False in boolean contexts and non-zero values to True. Having False evaluate to 0 and True to 1 in numeric contexts is however less intuitive (and somewhat useless):

>>> a = [1, 2, 3]
>>> a[True], a[False]
(2, 1)
>>> True + True # this one was rather unexpected...
2

2. The default argument values are evaluated at the point of function definition in the defining scope

This is probably one of the most frequent gotchas out there:

>>> def a(b=[]):
...     b.append(3)
...     print b
>>> a()
[3]
>>> a()
[3, 3]

The proper way to do this is to set b’s default value to None in the declaration and set it to [] inside the function body if it’s set to None:

>>> def a(b=None):
...     if b is None: b = []        
...     b.append(3)

3. Everything is an object

Although it’s a fact which may escape even the best of programmers at the beginning, Python is actually object oriented to the core. Despite the fact that it allows you to write procedural (and even functionalish) code, everything is an object. Functions are objects, data types are objects etc. This:

69.bit_length()

doesn’t work because the period is parsed as part of the numeric token (think of 69.j or 69.e1); this however:

(69).bit_length()

works. Being objects, functions can also have attributes:

>>> def a():
...    print a.x
>>> a.x = 3
>>> a()
3

This comes in handy e.g. for giving a decorated function the same internal name (for introspection purposes) as the original function:

def print_call_decorator(fun):
    def replacement(*args, **kwargs):
        res = fun(*args, **kwargs)
        print r'Call %s.%s => %s' % (inspect.getmodule(fun).__name__, fun.__name__, res)
        return res
    replacement.__name__ = fun.__name__
    return replacement

4. Generators, sets & dictionaries also have comprehensions (called “displays”)

As you probably know, list comprehensions are a great way to generate a new list from another iterable. But it goes further… Generators, sets and dicts also have something similar, called displays. The basic syntax is this:
generator = (value for ... in ...) dict = {key:value for ... in ...} set = {value for ... in ...}
E.g.:

>>> a = ['a', 'b', 'c']
>>> d = {x:a.index(x) for x in a}
>>> d
{'a': 0, 'c': 2, 'b': 1}
>>> d_rev = {d[x]:x for x in d}
>>> d_rev
{0: 'a', 1: 'b', 2: 'c'}

This makes reversing a dictionary for example much cleaner.
What makes displays even more fun are the *list* displays, which are essentially list comprehensions but with unlimited depth; using them to flatten a list of lists would look something like this:

flat = [x for y in lst for x in y]

The x/y order in the example is not a mistake; that’s actually the proper order.

5. GIL: Python threads aren’t

Not on multi-processor machines, at least. Yes, there is a threading module (aptly named), but due to the Global Interpreter Lock, threads of the same (Python) process can’t actually run at the same time. This becomes more of an issue when deploying native-Python servers, as they don’t get any benefit from the number of cores installed on the machine (drastically limiting the number of open sockets a Python process can handle at the same time as opposed to a native one written in C).

6. for has an else clause

…and so do the try…except/finally and while constructs. In all cases, the else branch is executed if all went well (break wasn’t called to stop cycles / no exception occurred). And while the else branch may be useful to perform when you want something to happen only if the cycle construct wasn’t “broken” (the classic example is handling the fact that the cycle hasn’t found the value it was looking for), try doesn’t really need an else clause, as the following are equivalent and the latter seems at least to me more readable:

with else:

try:
    this_may_crash()
except:
    handle_it()
else:
    call_me_if_it_didnt_crash()

without else:

try:
    this_may_crash()
    call_me_if_it_didnt_crash()
except:
    handle_it()

7. Tuple assignment order

In tuple assignments, the left-values are eval’ed & assigned in order:

>>> a = [1, 2, 3]
>>> i, a[i] = 1, 5 # i is set to 1 *before* a[i] is evaluated
>>> a
[1, 5, 3]

This happens because tuple assignments are equivalent to assigning the unpacked pairs in order; the second line above is therefore equivalent to:

>>> i = 1
>>> a[i] = 5

8. Scope juggling

Inside a function, variables are resolved in the global scope if no direct variable assignment appears in the
function, but are local otherwise (making Python clairvoyant, as it is able to tell that something is going to happen later on inside the function, i.e. a variable will be set). Note that attributes and sequence/dict values can still be set, just not the “whole” variable…

a1 = [1, 2, 3]
a2 = [1, 2, 3]
b = 3
c = 4
def fun():
    global b
    print c # crashes, because c is resolved to the local one (which is not set at this point)
    print b # works, because the global directive above forces b to be resolved to the global value
    a1[0] = 4 # works, because a1 is not directly set anywhere inside the function
    a2[0] = 5 # crashes, because a2 is later on directly set
    c = 10
    b = 11
    a2 = 'something else'

Bonus facts

As a bonus for making it through to the end, here are some lesser known / less frequently pondered upon facts about Python:

a reference to the current list comprehension (from the inside) can be obtained with:
```
locals()['_[1]'].__self__
```
list comprehensions can contain any number of for…in levels (this is actually documented). Flattening lists:
flat = [x for y in lst for x in y]
range() actually builds a list, which can be slow and memory-consuming for large values; use xrange()
modules have a dict attribute .__dict__ with all global symbols
the sys.path list can be tampered with before some imports to selectively import modules from dynamically-generated paths
flushing file I/O must be followed by an os.fsync(…) to actually work:
```
f.flush()
os.fsync(f.fileno())
```
after instantiation, object methods have the read-only attributes .im_self and .im_func set to the current object’s class and the implementing function respectively
some_set.discard(x) removes x from the set only if present (without raising an exception otherwise)
when computing actual indexes for sequences, negative indexes get added with the sequence length; if the result is still negative, IndexError is raised (so [1, 2, 3][-2] is 2 and [1, 2, 3][-4] raises IndexError)
strings have the .center(width[, fillchar]) method, which padds them centered with fillchar (defaults to space) to the length given in width
inequality tests can be chained: 1 < x < 2 works
the minimum number of bits required to represent an integer (or long) can be obtained with the integer’s .bit_length() method

Switching to Linux (1)

vtopan — Thu, 11 Nov 2010 02:56:55 +0000

Since my first Linux “experience” (which happened some eight years ago during a CS lab at UTCN), every couple of years I spend somewhere between a few days to several weeks trying to switch over to “the other side”. The reasons behind these (as of yet futile) attempts revolve mostly around the concept of “freedom”, and are beyond the scope of this writing. What I find of greater interest is the evolution of Linux-based OSes, and in particular of their target audience, which shifts more and more from hardcore enthusiasts willing to spend countless hours setting up a new machine toward average computer users (even the ones of the “point-and-clicky” variety).

The “sparse” look

My first Linux (Mandriva) was text-only; X-Window was in an “almost-working” state (crashing often, and even more often not being able to start at all) on most Linux machines I touched back then. Some time later most distributions had GUIs, but all the relevant work was still done beyond the scenes by console programs, which is still the case today. The myriad of “flavors” (and window managers) has made it practically impossible to write even remotely portable GUI interfaces for Linux, so graphic interfaces get “strapped” (pardon the low-brow hint; it truly feels like le mot juste) on console programs.

And that’s precisely how most Linux GUIs look and feel like: painful. Most of the window space is simply wasted: text is larger than necessary (for most people) and controls are separated by vast amounts of empty space, giving the interfaces a very “sparse” look. And deeper on the causality chain of problems is the fact that there simply isn’t that much exposed functionality in most Linux GUIs. Although to a much lesser extent than it was the case years ago, you still have to drill down way beyond the graphical interface in order to accomplish most non-trivial tasks. And then there’s responsiveness. After being spoiled by native graphical interfaces (some optimized to the point of writing machine assembly-level code) with excellent responsiveness (as it is more often than not the case on Windows), the sensible lag experienced on most interactions with Linux GUIs tends to annoy me in a very subtle manner. Then there’s Java and Java-based GUIs, which make Linux GUIs feel lightning fast, but I won’t go there.

Along came Ubuntu

My most recent attempt at Linux started a couple of months ago, but was interrupted by even-more-work-than-usual at the job, and would have been completely forgotten and abandoned if not for a Linux-vs-Windows themed conversation with my coworkers. I complained about most of the things that annoyed me about Linux (no decent music player is near the top of my list), and after getting many answers along the main theme of “software X has come a long way since then“, I decided to actually give it (yet) another chance.

My previous experience with Ubuntu (8.04 I think) had been almost pleasant, by far more so than any other previously-tried flavor (most notable being Mandriva back when it was called Mandrake and Red Hat at home and CentOS at work), so Ubuntu 10.10 felt like the way to go. After some research regarding “the most popular Linux”, Linux Mint popped up as a tempting Ubuntu/Debian-based alternative. The sheer volume of documentation / user-assistance available for the vanilla Ubuntu convinced me to stick with it, and so far it has been the right decision: as I’ve become accustomed when setting up a Linux OS, I’ve had problems requiring “workarounds” from the first day.

The good

In spite of the minor technical “misadventures” during setup, the Ubuntu 10.10 GUI finally feels mature (and almost responsive *enough*). The themes look good, the fonts are readable even at smaller sizes etc. And then there’s the repositories: thanks to my recently-acquired 100Mbit Internet connection, in a few hours after the installation I was already playing a pretty good-looking FPS (Assault Cube) and enjoying it. I’m not much of a gamer, but on one hand I was curious how far free games have come along, and on the other I had a lot of blood-spill-requiring-frustration left over from working out what should have been minor kinks and turned into major research themes.

I actually managed to set up both my PPP Internet connection and a VPN to my workplace without much hassle, which is a notable first. The VPN actually works better than on Windows because I have a convenient checkbox option to only route traffic going towards the VPN server’s network through it (as opposed to manually deleting the 0.0.0.0 route through the VPN server on Windows, because some clever bloke figured I *must* want *all* my Internet traffic to be routed through a gateway which only knows private addresses).

The bad

The NTFS driver (ntfs-3g). It’s not bad per se, in fact it also has “come a long way” and when it works, it works fine. But in one instance it *chose* not to work for me, which I found very frustrating and annoying. My problem (and it seems to be a rather common one) is that on a recently-acquired USB hard-disk Windows appears to have messed up either the partition table or the NTFS filesystem; the problem is that it only appears that way to the ntfs-3g driver. Which is not to say that it’s wrong (from what I could gather, the size of the filesystem is set to a larger value than there actually is room on the disk, the difference being of a few sectors = a few KB); it’s just that Windows doesn’t seem to mind and reads from/writes to the disk without problems. I imagine that if I were to write to those last few KB on the disk the data would be lost, but at least I can access the data on the disk, which ntfs-3g won’t allow, because it wouldn’t mount the disk even in read-only mode. Adding insult to injury, the “Tuxera CTO” (an otherwise friendly and helpful person) suggests (here) that the only solution to ignore the warning is to “change the sourcecode”. Booting back into Windows, backing up the data and reformatting the drive to a smaller size fixed the problem, but it shouldn’t have been necessary, and the “I know what’s right for you better than you ’cause I’m the pro” attitude was somewhat disappointing.

Another problem is the lack of a decent file manager. After using all the “commanders” (Norton Commander, then the amazing Dos Navigator and nowadays Total Commander), I’m used to having a software which can handle all file-related operations (and I do a lot of them for my day job) easily and efficiently. TC, which I wholeheartedly recommend on Windows, handles everything just fine. On Linux, so far I haven’t even been able to find a (GUI) file manager with an actual “brief” view mode; all of them insist on giving me a long line of information about each file, whether I actually need it or not, and waste about two thirds of the available screen space in the process. All the features offered by TC, not to mention the plethora of plugins available for it, are still far, far way. And since we’ve hit the sensitive point of software equivalents for Linux, here’s what I’ve managed to find so far.

Software alternatives for Linux

File manager

As mentioned above, I’m profoundly dissatisfied with what I’ve found so far. MC is a must, but lacks many features. Double Commander seems be the best contender, and is built to be similar to TC (going as far as plugin interchangeability, if only there were any ELF plugins for TC…), which is a plus.

Music player

After finding a decent music player (i.e. one which is stable and has a compact interface like WinAMP and the other *AMPs on Windows) has been a seemingly impossible feat for years, along came Audacious, and all became well.

Image viewer

If good file managers and music players are hard to come by in Linux, image viewers are far more challenging. Neither one seems to grasp the basic concept of viewing a folder of images; all of them insist on “organizing my collection of photos” (’cause it’s trendy to index collections of stuff), and offer either very cumbersome methods of simply browsing image folders, or simply no way at all (except for, of course, indexing/organising the folders into a collection). The excellent XnView image viewer for Windows has a multi-platform version aptly-called XnView MP, with the downside that development is favored for the Windows version and Linux builds don’t come for each version.

Other

I’m still looking into options for a development IDE (for C and Python in particular), with no luck as of yet.

As far as web browsing is concerned, all the relevant alternatives for Windows (Opera, Firefox and Chrome) are present on Linux, and from the order of the above enumeration my browser of choice should be obvious enough.

For an office suite I use OpenOffice on Windows, which is also available on most platforms.

I strongly recommend Guake as a terminal and Gnome Do as a generic application/document opening method.

[To be continued]

Mafia Wars tips, tricks & fixing problems

vtopan — Mon, 25 Jan 2010 19:33:09 +0000

Mafia wars

I, like many others, have fallen victim to Zynga’s Mafia Wars, the “game” that looks like just-another-text-based-massively-multiplayer-something-something not worth clicking into, and yet becomes highly addictive if given the chance (in fact, it’s how I got to have a Facebook account and the only actual use I’ve found for the afore-mentioned account). This is not a rant on it’s impressive success however, but a few useful things I’ve learned about the game.

Job help / boss defeat links

I got to writing this text because one of my Bangkok boss defeat links didn’t publish. And since that would’ve brought me ~500K – 1 million “Baht”, I *had* to figure out how to fix it. Since it might actually be of use to other MW-victims, here goes the info.

Your Facebook ID

You need your Facebook ID; hover over a link to your name/homepage anywhere on Facebook and get it from the link. The link should look something like: http://www.facebook.com/profile.php?id=your-facebook-ID&ref=nf

Bangkok – boss

Brawler (1st) episode:
http://apps.facebook.com/inthemafia/track.php?next_controller=story&next_action=claim_boss_bonus&zy_track=newsfeed&from=your-facebook-ID&cityId=4&jobId=37&ztrack_category=bangkok_boss_defeated&friend=your-facebook-ID&next_params=%7B%22target_id%22%3A%22your-facebook-ID%22%2C+%22job_city%22%3A%224%22%2C+%22job_id%22%3A%2218%22%7D&ref=nf
Criminal (2nd) episode:
http://apps.facebook.com/inthemafia/track.php?next_controller=story&next_action=claim_boss_bonus&zy_track=newsfeed&from=your-facebook-ID&cityId=4&jobId=37&ztrack_category=bangkok_boss_defeated&friend=your-facebook-ID&next_params=%7B%22target_id%22%3A%22your-facebook-ID%22%2C+%22job_city%22%3A%224%22%2C+%22job_id%22%3A%2237%22%7D&ref=nf

The Bangkok boss defeat links provide the clickers with B$1000 and 6 XP, and the publisher with $500.000-5.000.000 Baht, so they’re a great thing.

Cool tools

I’ve found some very useful tools for speeding up routine operations (collecting bonuses, sending gifts etc.) in Mafia Wars made by MW-enthusiasts. I’m not sure they respect Zynga’s TOS (actually I have a strong hunch they don’t), so use with care (and measure, to keep the game fun for everyone):

Spockholm Mafia Wars Tools (news and talk page here, beta versions here)
Josh Miller’s Bookmarklets

Some of the bookmarklets help you choose what items to buy to improve your attack/defense, find ongoing wars in which you might help or advise you on whom to promote to your Top Mafia to maximize the bonuses.

Misc hints

the Zynga toolbar (yes, like everyone else, I hate toolbars) is safe to install and provides an extra “mini energy pack” of 25% of your energy every 8 hours (it also gives an item of less use). To actually get the bonus you have to start MW by clicking the “Play Now” button on the toolbar when the last text on the toolbar says “Ready” in green (instead of a countdown timer in red). For some reason, I also have to login fresh (delete cookies) to Facebook every time for it to work.
Note: some methods have been published to get the mini pack without the toolbar; the loopholes have been patched, and if new ones will be found, they will be patched as well. Since it’s not that much of a hassle, you’re better off just installing the toolbar.
in Bangkok, the final Boss job gives a payout between $500.000-5.000.000 “Baht”, which can really help in starting your businesses, so make sure you publish it and enough friends help you
if you’re like me and only use your Facebook account for playing MW, you can find plenty of friends to add to your mafia in the comments on the MafiaWarsFans fan page; following that page can also keep you informed on game news and might get you extra Godfather points, offered there on random occasions by Zynga

Later edit

After Mafia Wars I played Travian, and then Lord of Ultima (which has amazing graphics for a browser game), and then I simply decided my time was pointlessly wasted on them, so I stopped. You can too!

Recovering data from a dead Windows (NTFS) disk using Linux

vtopan — Sun, 15 Nov 2009 23:51:11 +0000

At some point in your IT-enthusiast life you must’ve had at least one dead HDD, off of which Windows wouldn’t boot anymore. Up until a while ago, particularly if the partitions were formatted with NTFS, the situation was pretty much hopeless. Nowadays, with very-much-improved NTFS support under Linux (and rather tolerant to faults compared to its native counterpart under Windows), it isn’t always so. If the HDD is in a “coma” (i.e. almost dead, but not still “sort of” kicking), booting off a Linux live CD might still help recover (some of) the data. Basic steps:

Get a Linux live CD distribution which has good built in NTFS support (most of them have basic support by now) and ddrescue
Boot off the live CD and use ddrescue to get a binary image of each partition or mount the partition(s) and copy the files to a safe place
[If using the dd(rescue) approach] mount the images as drives under Windows and copy the files or be brave and mount the partition in a VM and try to actually boot it, at least as far as a command prompt (safe mode) or use a backup/partitioning tool to write the images to another disk

If you’re not paranoid about security (by nature or by job description), i.e. you don’t use EFS for your most sensitive data, you’re pretty much off the hook. If you’ve made the punishable-by-huge-amounts-of-pain mistake of using EFS and your disk crashed, as is my case, hope is as dimmed as the foresight of the folks who designed NTFS and used more than the actual user password to encrypt the data. As it turns out, to decrypt the files you need a certificate which can only be generated on the machine which encrypted the files, which is

Linux live CDs with NTFS support

I’ve tried SystemRescueCd, Trinity Rescue Kit, RIP Linux and plain vanilla Knoppix, and Trinity Rescue Kit appears to be the best: it has ntfstools / Linux-NTFS installed, and it didn’t hang on boot because of the failing HDD (other distros did). As a sidenote, I haven’t managed to boot the GUI (X) of any of the distros, as my laptop monitor/graphics card seems to be uncooperative with the standard drivers/VESA mode, but apart from the visual partition manager, everything works fine from the console anyway.

When choosing a distro, the main points to check are if it has the ntfs-3g driver (as recent a version as possible, as it keeps getting better at a fast pace) and the ntfstools / Linux-NTFS suite I mentioned earlier, especially if you’ve used EFS to encrypt your data (in which case the only viable solution appears to be ntfsdecrypt from that suite, which needs the certificate with which the files were encrypted, which in it’s turn needs you to boot the (dead) machine, but it appears to be the only way to get the data back).

Using dd/ddrescue to recover (NTFS) partitions

dd / ddrescue

The tool to move binary data from one place to another under Linux is dd. It also has a data-recovery-oriented cousin called ddrescue, which basically does the same thing, but is more fault-tolerant.
Basic dd usage:

dd if=/source of=/destination

if stands for input file and of for output file, and neither of them has to be an actual file (in the Windows sense); in the above example, /dev/sda1 is the first partition on the sda disk.
To back up just the MBR of the disk (the first 512 bytes) use:

dd if=/dev/sda of=/mnt/sdb1/saved/mbr.bin bs=512 count=1

This assumes that source disk is sda and that sdb1 is the partition to which you want to back up the data, so in your particular case they may need to be changed. See the next section if you’re not sure which disk is mapped to which name.
ddrescue uses fixed-position input (first) and output (second) arguments:

ddrescue -v /source /destination

The -v option makes ddrescue verbose (i.e. periodically print progress).
Note: by default, dd prints no progress/info until it’s job is finished. To check up on it’s progress, open another console (the terminals are mapped to Alt+N shortcuts in Linux, N >= 1, usually up to 4) and send it the USR1 signal. To do that, first you need to find it’s PID using ps:

ps -A|grep dd

Then, assuming the PID of the dd process is 3456, use kill:

kill -USR1 3456

That won’t actually kill the process, in spite of it’s name; it will just send it the USR1 signal, which makes dd print it’s current status (switch to the dd terminal to see it). The command’s name (“kill”) comes from it’s most frequent usage, which is to send a process the KILL signal (i.e. “kill” it).

Linux drive mapping

Linux maps your disks under /dev with names following the (“regex-like”) pattern [hs]d[abcd]. An h prefix means an (older) IDE disk, meanwhile an s prefix means a serial disk (usually an internal SATA or external USB disk). The individual partitions follow the disk naming + a digit to designate the partition number. So, for example, if you have a SATA disk with two partitions, the disk would be /dev/sda, the first partition would be /dev/sda1 and the second partition /dev/sda2.
To see the available disks/partitions, use ls (the Linux equivalent of dir):

ls /dev/sd*
ls /dev/hd*

To get extended disk info, use hdparm:

hdparm -I /dev/sda

The disks (actually the partitions) found under /dev need to be mounted before the files on them can be read/written; up until that point they are just huge blobs of binary data.

Note: for the rest of this writing, for simplicity’s sake, I’ll assume that sda is the broken disk and it has wto partitions, and that the recovered files/image go to sdb.

There are two ways to mount NTFS partitions: either using the default NTFS driver which comes with mount (ignores many problems, doesn’t care if Windows was improperly stopped & the drive was left “unclean”, read-only mode by default) or the ntfs-3g driver (more sensitive, read-write by default). Use the plain mount for the broken disk and the ntfs-3g version for the drives to which you need read-write access.
First off, you need to make appropriate folders for the partitions to be mounted under; standard practice is to do it under the /mnt folder. e.g.:

mkdir /mnt/sda1
mkdir /mnt/sda2
mkdir /mnt/sdb1

Note that the /mnt folder may not exist, in which case it must be created first: mkdir /mnt
Next, mount partitions from the broken disk (read-only):

mount /dev/sda1 /mnt/sda1
mount /dev/sda2 /mnt/sda2

The syntax of the mount command is straight-forward: mount /what /where; /what is the device, /where is the mount point in the filesystem. It takes other arguments, such as -t type to set the filesystem type, but NTFS is (nowadays) recognized automatically. The naming convention for the mount points is at your choice (you could mount the thing on something like /my/broken/disk/partition/number/1), but sticking to the “standard” /mnt path and using the original device’s name (or the partition letter if you’re more accustomed to that and a lazy typist, such as /mnt/c) is easier, and the help you find on the net will make more sense.
Last step in the mount process: mounting the destination disk in read-write mode (default for ntfs-3g):

ntfs-3g /dev/sdb1 /mnt/sdb1

mount -t ntfs-3g /dev/sdb1 /mnt/sdb1

The syntax is similar to the mount command; to check if the distro you chose has the ntfs-3g command built in, simply try to run it. If it doesn’t, choose another distro.

Copying the data

Run either dd or ddrescue (the latter is preferred if the disk is only partially readable):

dd if=/dev/sda1 of=/mnt/sdb1/saved/part1.bin

ddrescue -n /dev/sdb1 /mnt/sdb1/saved/part1.bin

WARNING: pay attention not to pass as the destination to dd/ddrescue entire disks unless you actually want their contents overwritten (which will be the case when you restore the saved image to a new disk); be sure to add a file name otherwise. The -n option prevents ddrescue from retrying error areas, which is usually what you want. If you have a disk which does yield data after enough retries, don’t use it.

Mounting the (NTFS) partition(s) from Linux/Windows

You can mount the newly backed-up partitions from Linux using the loop feature:

mount -o loop /mnt/sdb1/saved/part1.bin /mnt/part1

The partition can also be mounted directly from Windows using the ImDisk Virtual Disk Driver (free) or using some rather expensive commercial tools (google for alternatives).

Backing up/restoring partitions/whole disks

Alternatively, you can use the dd command to copy the entire disk and write the image to a fresh (identical) disk. Writing an image to a partition/disk using dd simply requires passing the disk as the of argument:
Restoring a partition:

dd if=/mnt/sdb1/saved/part1.bin of=/dev/sdc1

Restoring an entire disk:

dd if=/mnt/sdb1/saved/whole-disk.bin of=/dev/sdc

WARNING: be careful when overwriting raw partition/disk contents; choose other recovery methods unless you understand exactly what you’re doing.

Recovering files from raw data/deleted files: data carving

If the partition table/NTFS structure is broken and you can’t mount the partitions but you can read the binary data, you can use TestDisk to recover some of the files (the ones with a specific structure, such as images and music, are more likely to be found as opposed to, say, plain text files). This is basically the same thing that file recovery programs (such as Recuva) do on the unused space of a disk to recover deleted files.

Recovering EFS encrypted files

As I’ve mentioned in the opening paragraph, to recover EFS encrypted files, even under Linux, you need a recovery certificate. If you don’t have one, EFS file recovery software might help, but I’ve had little luck using them. I know of no open source/free software which does that, so you’ll probably have to use commercial software such as Advanced EFS Data Recovery from ElcomSoft (demo version available). The link called “encrypted file system recovery” from the following section details the process of manually extracting the required information for EFS recovery.

Moral of the story

ALWAYS BACK UP YOUR IMPORTANT DATA. Seriously. Now. Go get some storage space (USB flash drive, external hard disk, even DVDs if you make a new one often enough, as they tend not to last very long) and copy your data on it. GO!
Don’t use EFS under NTFS. Use an alternative encryption solution, e.g. TrueCrypt. There are portable (i.e. works-from-flash-drive) editions of most encryption tools should the need arise, and they are reliable (I’ve used TrueCrypt without problems for quite a while now).
If you MUST use EFS, create a recovery certificate using CIPHER /R:filename (details here) and store it in a safe place.

Karaoke songs & free player

vtopan — Tue, 06 Oct 2009 00:51:22 +0000

Karaoke is one of my more enjoyable means of killing time (and music itself, some might argue, as I’m a rather untalented singer). A while ago I’ve found a free karaoke player (far from perfect, as it BSODs my new laptop, but pretty usable otherwise), which together with some fun .kar songs I’ve gathered over time make for a ~20 MB pack of (potentially) great fun.

The player – Karafun

You can get a player called Karafun here (if you know of any other, preferably better, free karaoke player, lemme know). You don’t have to buy the songs from that site too; there are plenty of free karaoke (.kar) songs on the web. It’s also portable, meaning you can copy the installed folder (c:\Program Files\KaraFun by default) on a USB stick and carry it around.

The songs (.kar files)

The most commonly used song files for karaoke are .kar files, and there’s plenty of them online. The .kar format, for the inquisitive(ly?)-natured, is a MIDI (instrumental music) file with synchronized lyrics strapped on. This is relevant because some of the files you’ll find won’t have the .kar extension, but a .mid extension; they most likely will still work (the lyrics could be missing from such files, so try playing them to make sure they’re there).

Google for “free kar songs” or something similar and you’ll find many searchable databases of such songs. To find the .kar file of a particular song, google for it’s name + kar (e.g.: “sinatra my way kar”). In Opera, clicking links pointing to .kar files will play them instead of prompting to download them, so simply right click & “Save Linked Content As” to save them.

I’ve found a nice and well-organized collection on karaokebash.com, and a good .kar search engine on vanbasco.com.

Personal selection

As I’ve already mentioned, I’ve put together a collection of .kar songs, split in two “tiers”: “the most frequently played” and “other good karaoke songs”, plus a huge folder called “unsorted” which occasionally feeds the two categories above when I have enough time to plow through the pile.

Songs I (and my friends) most often sing along to (download link the whole pack from easy-share.com or wikiupload.com):
ABBA – Chiquitita.kar, ABBA – Fernando.kar, ABBA – Thank You For The Music.kar, Ace of Base – All That She Wants.kar, Aerosmith – Cryin’.kar, Afroman – Cause I Got High.kar, Andrea Bocelli – Con te partiro.kar, Andrea Bocelli – Vivo per lei.kar, Animals – House Of The Rising Sun.kar, Aqua – Barbie Girl.kar, Bangles – Manic monday.kar, Beach Boys – Kokomo.kar, Beatles – Hey Jude.kar, Beatles – Yesterday.kar, Bee Gees – Tragedy.kar, Berlin – Take My Breath Away.kar, Billy Joel – We Didn’t Start the Fire.kar, Blondie – Heart of Glass.kar, Bobby McFerrin – Don’t Worry, Be Happy.kar, Bon Jovi – Always.kar, Bon Jovi – Bed of Roses.kar, Bon Jovi – Blaze Of Glory.kar, Bon Jovi – Livin’ On A Prayer.kar, Bon Jovi – This Aint a Love Song.kar, Bonnie Tyler – It’s a Heartache.kar, Bonnie Tyler – Sometimes when we touch.kar, Bonnie Tyler – Total Eclipse Of The Heart.kar, Boyzone – Father and Son.kar, Boyzone – Words.kar, Brian May – Too much love will kill you.kar, Bryan Adams – Cloud number 9.kar, Bryan Adams – Everything I do, I do it for you.kar, Celine Dion – Beauty And The Beast.kar, Coolio – Gangstas Paradise.kar, Cranberries – Zombie.kar, Crash Test Dummies – Mmm.kar, Dean Martin – Sway.kar, Dean Martin – That’s Amore.kar, Disney’s Jungle Book (Terry Gilkyson) – Bare Necessities.kar, Don McLean – American Pie.kar, Eagles – Hotel California.kar, Eddy Grant – Gimme Hope Joanna.kar, Edith Piaf – Ne me quitte pas.kar, Elvis Presley – Are You Lonesome Tonight.kar, Elvis Presley – Can’t Help Falling In Love.kar, Elvis Presley – Love Me Tender.kar, Elvis Presley – Suspicious Minds.kar, Eric Clapton – Tears In Heaven.kar, Extreme – More Than Words.kar, Frank Sinatra – My Way.kar, Frank Sinatra – Something Stupid.kar, Frank Sinatra – Strangers In The Night.kar, G. Capurro – O sole mio.kar, Gerry & The Pacemakers – You’ll Never Walk Alone.kar, Gerry Rafferty – Baker Street.kar, Gloria Gaynor – I Will Survive.kar, Gordon Lightfoot – If You Could Read My Mind.kar, Guns’n'Roses – Sweet Child Of Mine.kar, Harry Belafonte – Banana Boat Song (Day-O).kar, Jim Croce – Operator.kar, Jimmy Buffett – Margaritaville.kar, Joe Cocker – You Are So Beautiful.kar, Joe Dassin – Et si tu n’existais pas.kar, Julio Iglesias – A veces si, a veces no.kar, Led Zeppelin – Stairway To Heaven.kar, Louis Armstrong – What A Wonderful World.kar, Lucio Dalla – Caruso.kar, Mariah Carey – Without you.kar, Meat Loaf – I’d Do Anything For Love.kar, Monty Python – Always Look On The Bright Side Of Life.kar, Mungo Jerry – In The Summertime.kar, Pet Shop Boys – Go West.kar, Phil Collins – Groovy kind of love.kar, Queen – Bohemian Rhapsody.kar, Queen – Love Of My Life.kar, Queen – Too Much Love Will Kill You.kar, Ray Charles – Hit the road Jack.kar, Richard Marx – Hazard.kar, Robbie Williams – Supreme.kar, Rod Stewart – Sailing.kar, Rolling Stones – Satisfaction.kar, Scorpions – Wind of change.kar, Smokie – Alice.kar, Soul Asylum – Runaway train.kar, Status Quo – In the army now.kar, Sting – Englishman in New York.kar, Supertramp – The Logical Song.kar, Suzanne Vega – Tom’s Diner.kar, The Carpenters – Close To You.kar, The Righteous Brothers – You’ve Lost That Lovin Feelin.kar, Tina Turner – The best.kar, Tom Jones – Delilah.kar, Tony Braxton – Unbreak My Heart.kar, Ugly Kid Joe – Cats In The Cradle.kar, Westlife – Seasons in the sun.kar, Whitesnake – Here I go again.kar, Whitney Houston – I will always love you.kar.

Listing the modules (DLLs) of the current process without API functions

vtopan — Wed, 27 May 2009 23:22:51 +0000

There is a simple way to walk the list of loaded modules (DLLs) of the current Windows process without calling any API functions. It also works with other processes, but it obviously involves reading the respective process’ memory space, which in turn involves using the aptly-named ReadProcessMemory function, defeating the whole purpose of not using APIs.

The PEB and the TIB

The PEB (Process Enviroment Block), briefly documented here (the page which passes as “documentation” on MSDN just… isn’t) is a structure (stored for each process in it’s own memory space) which holds various process parameters used by the OS such as the PID, a flag set if the process is being debugged, some localization information etc. The PEB is pointed to by the TIB (Thread Information Block), which is always located at FS:[0] (if that sounds like black magic, either pick up a book on assembly language and/or x86 processor architecture, or ignore that piece of information and use the code below to get to it).

One of the PEB entries is a list of, well, the loaded modules of the process. Actually, a PEB member points to a structure called LoaderData (it’s type being PEB_LDR_DATA), again “almost documented” by MS, which points to the list we’re interested in, looking something like this:

typedef struct LDR_DATA_ENTRY {
  LIST_ENTRY              InMemoryOrderModuleList;
  PVOID                   BaseAddress;
  PVOID                   EntryPoint;
  ULONG                   SizeOfImage;
  UNICODE_STRING          FullDllName;
  UNICODE_STRING          BaseDllName;
  ULONG                   Flags;
  SHORT                   LoadCount;
  SHORT                   TlsIndex;
  LIST_ENTRY              HashTableEntry;
  ULONG                   TimeDateStamp;
  } LDR_DATA_ENTRY, *PLDR_DATA_ENTRY;

It’s a linked list (the LIST_ENTRY structure contains the forward pointer called Flink) which contains serveral useful pieces of information about each module, among which it’s image base (BaseAddress), virtual address of the entrypoint (EntryPoint) (NOT the RVA you find in the PE header, but the actual computed VA) and DLL name with (FullDllName) and without (BaseDllName) the path, as UNICODE_STRINGs. The list is circularly linked and has a sentinel element with the BaseAddress member set to 0.

The list entries should be of the LDR_DATA_TABLE_ENTRY data type described by MS here, but the actual structure found in-memory doesn’t have the first member (BYTE Reserved1[2]). An alternative (and more complete) definition of the structure can be found here, but it has three LIST_ENTRY members at the beginning instead of just one.

Getting a pointer to the list

We retrieve the pointer to the list from the PEB using inline assembly from the C source code:

__declspec(naked)
PLDR_DATA_ENTRY firstLdrDataEntry() {
   __asm {
      mov eax, fs:[0x30]  // PEB
      mov eax, [eax+0x0C] // PEB_LDR_DATA
      mov eax, [eax+0x1C] // InInitializationOrderModuleList
      retn
      }
   }

There are actually three lists of modules, sorted by three different criterias; the one we’re using is sorted by the order in which the modules were loaded.

Example

Using the structure defined above and the firstLdrDataEntry function, it becomes trivial to walk the list of loaded modules:

void main() {
   PLDR_DATA_ENTRY cursor;
   cursor = firstLdrDataEntry();
   while (cursor->BaseAddress) {
      printf( "Module [%S] loaded at [%p] with entrypoint at [%p]\n",
              cursor->BaseDllName.Buffer, cursor->BaseAddress,
              cursor->EntryPoint);
      cursor = (PLDR_DATA_ENTRY)cursor->InMemoryOrderModuleList.Flink;
      }
   }

Using NtDeleteFile from Delphi

vtopan — Tue, 26 May 2009 15:43:39 +0000

The NtDeleteFile API

The native API NtDeleteFile performs the same task as the user-mode API DeleteFile, but interrestingly enough, the user-mode API does not call the native API to perform it’s task. As explained here, normally files are deleted through calls to NtSetInformationFile. The main difference in behavior comes from the fact that NtDeleteFile does not wait for handles on the file to close before deleting it (note that if the file is “open for normal I/O or as a memory-mapped file”, it still can’t be deleted, so only read-only handles will be ignored).

Data structures

The required structures (not defined in Delphi) are UNICODE_STRING and OBJECT_ATTRIBUTES.

Steps

The steps to remove the file are:

convert the “DOS” path name to an “NT” path name (basically prepend “\??\” to whatever the plain path is) using RtlDosPathNameToNtPathName_U;
fill in the OBJECT_ATTRIBUTES structure;
call NtDeleteFile.

Importing the APIs

Using the afore-linked-to type definitions, we still need to import the native APIs:

function NtDeleteFile(ObjectAttributes:POBJECT_ATTRIBUTES):DWORD; stdcall; external 'ntdll.dll';
function RtlDosPathNameToNtPathName_U(DosName:PWChar; var NtName:UNICODE_STRING; DosFilePath:PPChar; NtFilePath:PUNICODE_STRING):BOOL; stdcall; external 'ntdll.dll';

Do note that this statically links the imported functions, making the whole application unable to load if the undocumented APIs are not present on the system (at least for Windows 2000 and XP they should be).

Converting DOS paths to native paths using RtlDosPathNameToNtPathName_U

Converting the DOS path name to a native path is done by the RtlDosPathNameToNtPathName_U API, which takes in a PWChar argument containing the DOS path and a pre-allocated UNICODE_STRING structure of MAX_PATH WideChars and returns True if it has successfully converted the path.

Source code

The code looks something like this:

function _DeleteFile(fileName:string):DWORD;
var
   oa:OBJECT_ATTRIBUTES;
   ws1, ws2:WideString;
   us:UNICODE_STRING;
begin
   Result := $C0000001; // STATUS_UNSUCCESSFUL, "generic" error 
   ws1 := fileName; // automatic String -> WideString conversion
   SetLength(ws2, MAX_PATH);
   us.Length := MAX_PATH;
   us.MaximumLength := MAX_PATH;
   us.Buffer := @ws2[1];
   if not RtlDosPathNameToNtPathName_U(@ws1[1], us, nil, nil)
      then Exit;
   oa.Length := SizeOf(OBJECT_ATTRIBUTES);
   oa.RootDirectory := 0;
   oa.ObjectName := @us;
   oa.Attributes := $40; // case insensitive
   oa.SecurityDescriptor := nil;
   oa.SecurityQualityOfService := nil;
   Result := NtDeleteFile(@oa); // pass on the NTSTATUS
end;

How to check if a thread/process is suspended (get thread state)

vtopan — Wed, 15 Apr 2009 01:07:18 +0000

The basic steps to get to a thread’s status information is the following (knowing of course both the process ID (hence forth PID) and the thread ID (TID)):

call NtQuerySystemInformation with SystemInformation set to SystemProcessInformation (5)
iterate over the array of SYSTEM_PROCESS_INFORMATION structures (the structure contents is (wrongfully) explained here; correct version here) to find your PID (ProcessId member) of interest
iterate over the array of SYSTEM_THREAD structures (detailed below) to find the desired TID (UniqueThread member) and check the State and WaitReason members; both must be set to 5 if the thread is suspended, any other values otherwise

As it’s probably obvious to most people keen on system-level programming, a process is suspended when all it’s threads are suspended, so all of them must be checked for the suspended status.

Step one: calling NtQuerySystemInformation

The required structures are defined here (for Delphi). The function isn’t defined in any headers, so we must declare it’s prototype ourselves:

function NtQuerySystemInformation(SystemInformationClass:DWORD; SystemInformation:pointer; SystemInformationLength:DWORD; ReturnLength:PDWORD):cardinal; stdcall; external 'ntdll';

Example usage:

var 
   spi:PSYSTEM_PROCESS_INFORMATION;
   size:DWORD;
begin
if (NtQuerySystemInformation(5, nil, 0, @size) = STATUS_INFO_LENGTH_MISMATCH) // SystemProcessInformation
   and (size > 0)
   then begin
        GetMem(spi, size);
        if NtQuerySystemInformation(5, spi, size, @size) = 0
           then begin
                [...] // do something with spi
                end
           else HandleError; // failed listing processes!
        FreeMem(spi);
        end
    else HandleError; // failed listing processes!
end;

HandleError is a fictional function (which you’ll most likely skip, ’cause you’re in a hurry to get things done, right? ).

Step two: iterating the process list

The structure only looks like a linked list item; the NextEntryOffset member is an actual offset from the beginning of the current structure to the beginning of the next one. This is needed because of the variable size of the structure (given by the variable number of threads for each process). We need an extra crt:PSYSTEM_PROCESS_INFORMATION variable to walk the pseudo-linked list because we must keep the original psi pointer to free it’s memory.
The outline of the code which iterates the processes looking for a PID (given the spi:PSYSTEM_PROCESS_INFORMATION pointer from above) would look like this:

var 
    crt:PSYSTEM_PROCESS_INFORMATION; 
[...]
    crt := spi;
    repeat
        if crt^.ProcessID = PID
           then begin
                [...] // do something with crt^
                break;
                end;
        crt := Pointer(DWORD(crt) + crt^.NextEntryOffset);
    until crt^.NextEntryOffset = 0;

Step three: find the appropriate thread

Given the ThreadInfo array in the structure located at the previous step, we iterate through it and test the State and WaitReason members for the item matching our TID:

var
    j:integer;
[...]
    for j := 0 to crt^.NumberOfThreads-1 do
        begin
        if crt^.ThreadInfo[j].UniqueThread = TID
           then begin
                if crt^.ThreadInfo[j].WaitReason = 5
                   then [...] // the thread is suspended
                   else [...]; // the thread is not suspended
                break; 
                end;
        end;

The State member must also be set to 5 (“waiting”), but if the WaitReason is non-null, the State must be 5 (and vice-versa), so there’s little point in checking it explicitly.

Additional info: thread starting address, priority etc.

If you’ve paid any attention while reading the structures, you might have noticed additional interesting information about threads and processes, such as the creation time, image path, priority, handle count and memory and I/O usage/history for processes (this is how Process Explorer gets, for example, the WorkingSet/PeakWorkingSet and ReadBytes/WriteBytes/OtherBytes information) and starting address, priority/base priority and various timing information for threads. The starting address is particularly interesting, because the NtQueryInformationThread API with ThreadInformationClass set to ThreadQuerySetWin32StartAddress (9) only works (on Windows pre-Vista) “before the thread starts running” (quoted from MSDN), which seems to me rather pointless in the first place.

The NtQuerySystemInformation API is also a useful replacement for the CreateToolhelp32Snapshot suite, yielding more information about processes and threads.

Top ten striptease songs

vtopan — Mon, 06 Apr 2009 01:36:08 +0000

Some while ago, when the opportunity arose to put them to good use, I realised I had rather few striptease songs sorted out. The very next day (my memories are rather foggy given circumstances, but it most likely was the next day, so we’ll work with that assumption) I set on a quest for the web’s favorite striptease songs. The results, however, were somewhat disappointing; mostly redundant and not as insightful as I had hoped. But plowing through them I also found some good (enough) ones, and managed to collect sufficient songs to tire out even some of the most dedicated strippers out there (not that that would be the purpose of the whole thing).

Stumbling over the sorted-out-music folder and running into them again, I decided to share them with the world, if only to spare other don quixotesque music fans out there from the pain of having to go thorough endless pages of search results only to find (mostly) the folowing songs listed over and over.

Top ten strip songs

Note: the order is mostly a matter of personal taste & current state of mind. The purpose is to provide you with a list of good striptease songs, rather than to make statements about the songs’ quality when compared to one another, as I find judging taste to be pointless.

#10. Kylie Minogue – Chocolate [watch]
Not much surprise in Kylie showing up in this list, now is there?

#9. Tina Turner – Private Dancer [watch]
Striptease 101.

#8. Pussycat Dolls & Busta Rhymes – Don’t Cha [watch]
Excellent rythm; rap & hip hop fans will appreciate it.

#7. Def Leppard – Pour Some Sugar on Me [watch]
Rock.

#6. Right Said Fred – I’m Too Sexy [watch]
Funny rather than sexy, but still a good song to drop clothes to.

#5. Joe Cocker – You Can Leave Your Hat On [watch]
Synonymous with “striptease song”; would be higher in the top if it weren’t so damn everywhere.

#4. Alannah Myles – Black Velvet [watch]
Another classic, slower & more sensual than most in this top.

#3. Prince – Cream [watch]
I’m not that much into Prince, but this song is by far among the best strip songs I’ve heard so far.

#2. Patricia Kaas – Mademoiselle chante le blues [watch]
French strip song. ’nuff said.

#1. En Vogue – Beat of Love [listen]
The bass drum rythm on this song is simply amazing, earning it a #1.

Bonus song

Instrumental music is excellent for striptease, and one of the best ones at that is the theme song from A Shot in the Dark (1964; the second Pink Panther movie, featuring Peter Sellers – a “dark” comedy which you may also enjoy) by Henry Mancini. If you need further proof, check out Ursula Martinez’ Hanky Panky (strip) show. The sax, as on Mademoiselle chante le blues, is excellent. On the instrumental topic, half funny-half sexy is also The Stripper from Joe Loss.

Runners up

As with most tops, some runners up got left out. In alphabetical order:

Ella Fitzgerald – I Just Wanna Make Love to You
Firefox – Sex Shooter
Ginuwine – Pony
Lovage – Sex (I’m A)
Paula Cole – Feelin’ Love
Queen – Fat Bottomed Girls
Sam Brown – Stop
Santana & Rob Thomas – Smooth
Shaggy – Hey Sexy Lady
The Sugababes – Push the Button
Tom Jones – Sex Bomb
Tom Jones – You Don’t Have to Be Rich
Touch & Go – Tango in Harlem
Zuchero – Baila Sexy Thing

You can sample most of them on ~~imeem.com~~ [later edit: imeem is now defunct; changed links to point to Google video searches], on youtube or simply googling/yahoo(ing?) for them.

Translating headers: struct / record field alignment in C / Delphi

vtopan — Tue, 10 Mar 2009 23:37:03 +0000

If you’ve ever taken a shot at porting headers from C to Delphi, you may have noticed the different way in which the struct/record fields are aligned, but most likely you havent. And if you had the bad luck of stumbling over a struct that gets aligned differently when ported to Delphi, even though the data types were translated correctly, you’re probably interested in what happens behind the scenes and how to fix it. Well, here it goes.

Why variable addresses are aligned in memory

As explained in depth here, because of it’s internal structure, the time required by the CPU to access data from the RAM depends on the alignment of the address it requests (the “aligned” memory addresses being accessed faster). Aligned means that the address should be a multiple of the CPU’s memory word (not to be confused with the WORD data type in C/Delphi, which is always 2 bytes), which is the least amount of memory the CPU can get from RAM at one time. It’s also the size of the general purpose registers on the CPU, and it represents the “chunk” of data the CPU is most “comfortable” working with. On x86 architectures it’s 32 bits, which means 4 bytes (on 64 bit architectures it’s… well, 64 bits = 8 bytes). So, the memory addresses should be multiples of 4 on x86 systems for faster access.

Nitpicking corner: why is the WORD type 2 bytes long then, if the memory word on x86 is 4 bytes? Well, it’s not actually *all* the x86 architectures that have a 4 byte memory word, just the 386-and-higher ones (the aptly-named 32-bit architectures). But back when dinosaurs still roamed the Earth and “640K of memory were enough for anybody” (to paraphrase (?) one of the foremost computer visionaries), the computer word was just 16 bits, or 2 bytes, and that’s when the WORD data type was coined (without very much foresight, truth be told).

Struct field alignment in C

According to this paragraph on Wikipedia, struct member fields are aligned based on their size by padding them with, basically, their size in bytes. So, assuming we declare the structure:

typedef struct { BYTE a; WORD b; BYTE c; DWORD d; } foo;

you might be tempted to assume it will take 8 bytes of memory. But, due to the afore-mentioned alignment, b will be aligned on a 2-byte boundary, and c on a 4-byte boundary, so that structure will actually be equivalent to this one:

typedef struct { BYTE a; BYTE padding_for_b[1]; // 1-byte padding to align .b WORD b; BYTE c; BYTE padding_for_d[3]; // 3-byte padding to align .d DWORD d; } foo;

and take precisely 12 bytes. You can force C’s hand not to align the members using the #pragma pack(1) directive, but (a) the code will be slower and (b) you can’t do that to standard structures in the Windows API for obvious reasons.

Record field alignment in Delphi

As explained in the “align fields compiler directive” help entry, in Delphi the record field alignment is not based on the field’s data type size, but on a constant specified by the ALIGN (A) directive (which can also be set for the entire project from Project->Options->Compiler->Record field alignment.
The default is {$ALIGN 8} (short: {$A8}), which means alignment to an 8-byte boundary. However, that’s how things work only in theory. In practice, for some mysterious reason, Delphi does exactly what C does: unless the alignment is set to 1 (eg. by {$A1}), it aligns record fields based on their size.

The difference

But if Delphi behaves just like C, why do things sometimes still crash? They do because of one exception to the rule: enumerations are treated differently. In C, they are 4 bytes in size by default (this can be changed using the #pragma enum directive). In Delphi, the enumeration size is controlled by the $Z directive, and by default it’s set to 1. Therefore, in order to get the proper translation of API structures from C to Delphi, you should use the {$Z4} directive.