Solved Using QImage.constBits(), RGB_888, and the actual pixel format seems wrong.

[enjoysmath]

def subimageSearch(img, subimg, center=False):
        img = img.convertToFormat(QImage.Format_RGB888)
        subimg = subimg.convertToFormat(QImage.Format_RGB888)
        x = c_int(-1)
        y = c_int(-1)
        xptr = addressof(x)
        yptr = addressof(y)
        # ptr = ctypes.cast(addr, INTP
        imgBuf = c_void_p(img.constBits().__int__())
        subBuf = c_void_p(subimg.constBits().__int__())
        w = c_int(img.width())
        h = c_int(img.height())
        sw = c_int(subimg.width())
        sh = c_int(subimg.height())
        _libc.subimageSearch(imgBuf, w, h, subBuf, sw, sh, xptr, yptr)
        if not center:
                return x.value, y.value
        return int(x.value + subimg.width() / 2 + 0.5), int(y.value + subimg.height() / 2 + 0.5)

subimageSearch(QImage("screenshot.png"), QImage("red_pixel.png"))

With the attached "screenshot.png" and "red_pixel.png" (a 4x4 red square made in MS Paint).

The bytes that the debugger is showing for both img and subimg are:

subimg	0x0621a538 "í\x1c$í\x1c$\x17\x6í\x1c$í\x1c$!\x6q¸‘ý\\\x2"	unsigned char *
    img	0x0621b768 "ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿí\x1c$í\x1c$ÿÿÿÿÿÿí\x1c$í\x1c$ate"	unsigned char * 

The y's with umlaut over them are correct as that is the character 255, and those pixels are indeed white. But the rest doesn't make since: the pixels should be pure red: 255, 0, 0 = rgb.

The debugger is on the Visual Studio side while my calling app is on the python side, and I'm using c_types to convert the types between python and the DLL function arguments, but that shouldn't matter.

Any idea what is going on ?

Here are teh images (they're really small):

[kenchan]

@enjoysmath Can't see any screen shots??

[SGaist]

@kenchan The two small red dots just above the miniature of your avatar at the bottom of the post.

[kenchan]

@SGaist Aha is see them now.

[enjoysmath]

This might have to do with BMP format's row padding to a multiple of 4-bytes but that doesn't explain how red comes up as far from 255,0,0

[enjoysmath]

It had to do with the padding. See wikipedia article on BMP format which I believe is used as the memory representation of pixels. Also, check mspaint and make sure you're using pure Red, I was not... -_- Save as BMP and double check using a hex editor on the file.

Here's my pixel search code if anyone needs it:

// SubimageSearch.cpp : Defines the exported functions for the DLL application.
//

#include "stdafx.h"
#include "SubimageSearch.h"

// Works with format RGB888

struct Pixel {
public:
	unsigned char r, g, b;
public:
	Pixel() {
		r = 0;  g = 0;  b = 0;
	}

	Pixel(unsigned char* img, int w, int x, int y)
	{
		int pad = (4 - ((w * 3) % 4)) % 4;
		unsigned char* px = &img[(w * y + x) * 3 + y * pad];
		r = px[0];  g = px[1];  b = px[2];
	}

	bool operator == (const Pixel& px)
	{
		return r == px.r && g == px.g && b == px.b;
	}
};

// This is an example of an exported function.
extern "C" {
	__declspec(dllexport) void subimageSearch(unsigned char* img, int w, int h, unsigned char* subimg, int sw, int sh, int* foundX, int* foundY)
	{
		Pixel px, px1 = Pixel(subimg, sw, 0, 0);

		for (int y=0; y < h - sh + 1; y++) {
			for (int x = 0; x < w - sw + 1; x++) {
				px = Pixel(img, w, x, y);

				if (px == px1) {
					for (int y1 = 0; y1 < sh; y1++) {
						for (int x1 = 0; x1 < sw; x1++) {
							px = Pixel(img, w, x + x1, y + y1);
							px1 = Pixel(subimg, sw, x1, y1);

							if (!(px == px1)) {
								goto not_found;
							}
						}
					}
					*foundX = x;
					*foundY = y;
					return;

				not_found:
					px1 = Pixel(subimg, sw, 0, 0);
					continue;
				}
			}
		}
	}
}

Notice the pad computation added in.