Truus DXP-Server

[BertTuyt]

Truus is a welknown program and used by infinite number of people.

As I know it is also used by many programmers, to test Truus against their programs.
Unfortunately Truus did not implement the DamExchange Protocol , as developed by Frank Mesander.
Does this mean this is totally impossible...... ????

Recently I started a discussion with Ed, where I shared some ideas with him, based on some assumptions and some educated guess-work.

And the result........ ????

It is possible, in theory , and ......................
It is possible, in practice, and .....................

We developed, and tested, a Truus DXP Server, which enable other DXP clients to play a match against Truus without any user-interference.

And the result, .........................

Wait for some further news from Ed.

Yours,

Bert

[wellnesswrotter]

Very good progress!
I am still struggling in implemeting the DXP with Frank Mesanders DLL for Visual Studio 2005. I am not familiar with C++, and the parameters are (for now) to difficult for me to guess.

Who wants to help with some code?

P.S. The next step for Truus DXP Server kan be a Webservice.
As implemented by me at :

http://www.snukenkuzco.nl/playgame/game ... tsBestMove

[Ed Gilbert]

Very good progress!
I am still struggling in implemeting the DXP with Frank Mesanders DLL for Visual Studio 2005. I am not familiar with C++, and the parameters are (for now) to difficult for me to guess.

Who wants to help with some code?

Johan, send me an email with the details of where you are having difficulty. I will help you.

-- Ed

[Ed Gilbert]

We developed, and tested, a Truus DXP Server, which enable other DXP clients to play a match against Truus without any user-interference.

Indeed, Bert has reverse engineered enough of the truus user interface to find the key control points, which allows another program to send it commands to move, set up arbitrary positions, command it to start a new game, set which color truus plays, and (very importantly) to read the truus board position, so that another program can determine when truus has made its move and what the move is. After testing that his ideas really do work, it was not very difficult to write a truus dam exchange server. I got the server working last week and am now able to run completely automated engine matches between kingsrow and truus. I am in the process of running a set of three matches, each one consisting of 158 games. The purpose is to get a baseline measure of the relative strength of the two programs. Of course I've already played a fairly large number of games manually between kingsrow and truus so I already had a pretty good idea of their relative strengths. But those games were played over a long period of time during which I had made a lot of changes in kingsrow. These three matches will give a snapshot of the performance right now, and serve as a baseline for future testing when I make changes in the program. Also, I think there is a perception of many people that truus is the strongest draughts program, and so we will see if that is really the case :-) I ran the first match last weekend, the second one is running now, and when it finishes I will start the third. I will post all results when the third is finished, which should be two or three days from now. Both programs are running on the same computer and using the same time controls.

-- Ed

[FeikeBoomstra]

That's really great! But I am not sure I understand the technical details. Is is true:

First Bert disassembled TRUUS.exe, found the hooks and then you added code to translate these hooks to DXP and then you create from the disassembled TRUUS a TRUUS.obj file, compile your c-program, and link the two together to obtain TRUUS_with_DXP???

Anyway, a great job. Are you also thinking of a way to share it? As TRUUS is a commercial program I don't want to be involved in hacking it, but I have a legal copy, and would like to transform it to an DXP version.

[Ed Gilbert]

That's really great! But I am not sure I understand the technical details. Is is true:

First Bert disassembled TRUUS.exe, found the hooks and then you added code to translate these hooks to DXP and then you create from the disassembled TRUUS a TRUUS.obj file, compile your c-program, and link the two together to obtain TRUUS_with_DXP???

Anyway, a great job. Are you also thinking of a way to share it? As TRUUS is a commercial program I don't want to be involved in hacking it, but I have a legal copy, and would like to transform it to an DXP version.

Hi Feike,

Technically it is not quite as you described. I wrote a program which loads and executes wintruus.exe, and then while it is running my program reads and writes in the memory space of the truus process. The locations that it reads and writes are some key variables in the truus GUI where it stores its side to move, board array, command buffer, etc. So then just add dam exchange to this program and you have a truus dam exchange server.

I will let Bert share the specific details of the truus control since that is his work.

I don't see anything legally or ethically wrong with this. I also have a legal copy of truus, and an awfully overpriced expensive legal copy it is too!

-- Ed

[FeikeBoomstra]

OK, I understand. So it is more like on the spot linking. In runtime you calculate the memory localtions to change/read.

[wellnesswrotter]

something like this:

Quick Memory Editor is a powerful game cheating tool that can search and edit game data in memory easily. It can search and edit any type of game data such as integer, floating point number, string and uncertain data like blood in game King of Fights. You can use it to toggle infinite health, ammo and lives to make you like a superman in your games. With a few clicks, you can easily build a game trainer and share it with other game players. It also has a memory hex editor for experienced users who are familiar with hex and PC memory knowledge.

[BertTuyt]

Herewith some background info related to the Truus DXP-Server.
First of all the original Truus is totally unaffected, no linking, compiling, patching or whatever.

The first step the TDXPS (Truus DXP Server) does is to Create a Process

Code: Select all

bool load_truus(HANDLE *handle_truus)
{
	DWORD dwSize = sizeof(TOKEN_PRIVILEGES);
	PROCESS_INFORMATION piProcInfo;
	STARTUPINFO siStartInfo;

	// Now create the child process.
	siStartInfo.cb = sizeof(STARTUPINFO);
	siStartInfo.lpReserved = NULL;
	siStartInfo.lpDesktop = NULL;
	siStartInfo.lpTitle = NULL;
	siStartInfo.dwFlags = STARTF_USESTDHANDLES;
	siStartInfo.cbReserved2 = 0;
	siStartInfo.lpReserved2 = NULL;
	siStartInfo.hStdInput = NULL;
	siStartInfo.hStdOutput = NULL;
	siStartInfo.hStdError = NULL;

	if (!CreateProcess(	
				NULL,
				TRUUS_FILENAME ,// command line
				NULL,			// process security attributes
				NULL,			// primary thread security attrs
				true,			// handles are inherited
				DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP,
				NULL,			// use parent's environment
				NULL,
				&siStartInfo,	// STARTUPINFO pointer
				&piProcInfo)) { // receives PROCESS_INFORMATION

		print_system_error("CreateProcess");
		return true;
	}
	*handle_truus = piProcInfo.hProcess;

	/* Wait for truus to initialize. */
	Sleep(2000);
	return false;
}

This starts Truus as a Child Process, and the "owner" TDXPS is now able to access Truus Memory via the handle_Truus.

How this is done, will be part of the next item.

Bert

[BertTuyt]

TDXPS, Part II

Now Truus as a process is running, and the TDXPS has a Handle to the Truus Memory.
Within Windows there are 2 important functions to Read/Write Memory in another process:

Code: Select all

ReadProcessMemory Function

Reads data from an area of memory in a specified process. The entire area to be read must be accessible or the operation fails.

Syntax

BOOL WINAPI ReadProcessMemory(
  __in   HANDLE hProcess,
  __in   LPCVOID lpBaseAddress,
  __out  LPVOID lpBuffer,
  __in   SIZE_T nSize,
  __out  SIZE_T *lpNumberOfBytesRead
);
Parameters
hProcess [in] 
A handle to the process with memory that is being read. The handle must have PROCESS_VM_READ access to the process.

lpBaseAddress [in] 
A pointer to the base address in the specified process from which to read. Before any data transfer occurs, the system verifies that all data in the base address and memory of the specified size is accessible for read access, and if it is not accessible the function fails.

lpBuffer [out] 
A pointer to a buffer that receives the contents from the address space of the specified process.

nSize [in] 
The number of bytes to be read from the specified process.

lpNumberOfBytesRead [out] 
A pointer to a variable that receives the number of bytes transferred into the specified buffer. If lpNumberOfBytesRead is NULL, the parameter is ignored.

Return Value
If the function succeeds, the return value is nonzero.

If the function fails, the return value is 0 (zero). To get extended error information, call GetLastError.

The function fails if the requested read operation crosses into an area of the process that is inaccessible.

Code: Select all

WriteProcessMemory Function

Writes data to an area of memory in a specified process. The entire area to be written to must be accessible or the operation fails.

Syntax

BOOL WINAPI WriteProcessMemory(
  __in   HANDLE hProcess,
  __in   LPVOID lpBaseAddress,
  __in   LPCVOID lpBuffer,
  __in   SIZE_T nSize,
  __out  SIZE_T *lpNumberOfBytesWritten
);
Parameters
hProcess [in] 
A handle to the process memory to be modified. The handle must have PROCESS_VM_WRITE and PROCESS_VM_OPERATION access to the process.

lpBaseAddress [in] 
A pointer to the base address in the specified process to which data is written. Before data transfer occurs, the system verifies that all data in the base address and memory of the specified size is accessible for write access, and if it is not accessible, the function fails.

lpBuffer [in] 
A pointer to the buffer that contains data to be written in the address space of the specified process.

nSize [in] 
The number of bytes to be written to the specified process.

lpNumberOfBytesWritten [out] 
A pointer to a variable that receives the number of bytes transferred into the specified process. This parameter is optional. If lpNumberOfBytesWritten is NULL, the parameter is ignored.

Return Value
If the function succeeds, the return value is nonzero.

If the function fails, the return value is 0 (zero). To get extended error information, call GetLastError. The function fails if the requested write operation crosses into an area of the process that is inaccessible.

So basically these 2 functions is all you need, and these are frequent used in the TDXPS.

Bert

[Ed Gilbert]

I got the server working last week and am now able to run completely automated engine matches between kingsrow and truus. I am in the process of running a set of three matches, each one consisting of 158 games. The purpose is to get a baseline measure of the relative strength of the two programs.

The three automated matches between kingsrow and truus have completed. The matches were run on a 2.4GHz Intel quad-core computer. Each program was set for match time control to make 75 moves in 9 minutes. If a conclusive result was not determined after 75 moves, the game result was recorded as "unknown". A conclusive result required a search score of "database win", "database loss", or 6 consecutive seaches returning "database draw".

In each match, 2-move ballots were used as start positions in order to avoid duplicate games. There are 81 ballots, but in two of these (31-27 16-21 and 32-27 16-21) black loses a man immediately so these are excluded. For each of the 79 remaining ballots, kingsrow first played the white side and then truus played the white side, so 158 games total.

Truus was given the same settings for all games: 160mb hashtable and 6-piece endgame databases. Kingsrow was given different settings in each match. In the first match, it was configured to use all of its endgame databases (full 2-7 pieces, partial 8 and 9 pieces) and parallel search was turned on. Although the computer is a quad-core, kingsrow was restricted to use only 3 search threads, since with pondering both engines are always searching. In the second match, kingsrow was set to identical settings as truus -- 6-piece endgame databases and single threaded search. In the third match, kingsrow was restricted to using only a 4-piece endgame db and single threaded search.

Here are the results:

Match 1: kingsrow/9pc vs. truus: 36 wins, 1 losses, 121 draws, 0 unknowns
Match 2: kingsrow/6pc vs. truus: 31 wins, 1 losses, 121 draws, 5 unknowns
Match 3: kingsrow/4pc vs. truus: 22 wins, 4 losses, 118 draws, 14 unknowns

After the match, the games with unknown results from matches 2 and 3 were analyzed using the 9-piece db, and for most of these a conclusive result was determined. The adjusted results are:

Match 1: kingsrow/9pc vs. truus: 36 wins, 1 losses, 121 draws, 0 unknowns
Match 2: kingsrow/6pc vs. truus: 33 wins, 1 losses, 122 draws, 2 unknowns
Match 3: kingsrow/4pc vs. truus: 22 wins, 5 losses, 131 draws, 0 unknowns

The only thing that surprised me was the result using the 4pc db. I expected that to be a lot closer.

-- Ed

[Rein Halbersma]

The adjusted results are:

Match 1: kingsrow/9pc vs. truus: 36 wins, 1 losses, 121 draws, 0 unknowns
Match 2: kingsrow/6pc vs. truus: 33 wins, 1 losses, 122 draws, 2 unknowns
Match 3: kingsrow/4pc vs. truus: 22 wins, 5 losses, 131 draws, 0 unknowns

The only thing that surprised me was the result using the 4pc db. I expected that to be a lot closer.

-- Ed

Ed, congratulations on these fantastic results!

This clearly shows that Kingsrow's strength is not primarily its gigantic databases but that the program's search and eval are first class as well.

Secondly, it is a big mental boost for the rest of us programmers since even without duplicating your enormous efforts in db-building, one might still be competitive with Kingsrow if one "only" has a better search and eval.

Thirdly, this (and all the many other matches that you played before) shows that automated testing is a very productive way of improving a program compared to manually adding knowledge (with or without the consultation of a GM).

Finally, these results prove that a tournament is not the proper format to decide relative program strengths (although tournaments might be fun in their own right). Lengthy automated matches are needed to determine such verdicts on a meaningful basis.

Rein

[BertTuyt]

TDXPS, episode III

Basically it is not required to apply Truus reverse-engineering (debug and/or disassemble) to get an clue regarding how to control Truus via an external DXP-Server (TDXPS). All you need is a program to examine the Process memory and next to that generate some ideas, assumptions and educated guesses, that's all.

Fortunately there is a good program available including description and source which guide you in this process.

Just have a look at Codeproject , article "Performing a hex dump of anothers process's memory", http://www.codeproject.com/KB/threads/MDumpAll.aspx

You can also download the Programm MDUMP, which is able to dump the memory image of an application once its loaded in memory. Selecting a specific program is possible with a click on its window.

So the only thing you have to do know, is to go to perform a Hex-dump of all memory pages, and find the right relevant locations

Bert

[Ed Gilbert]

Rein, thank you for your comments. But truus is an old program using 15-year old (or 20?) technology. It has a slow, shallow search, poor time management, and no opening book (or if it has an opening book nobody can figure out how to turn it on). I don't think it is a surprise to any of the programmers here that a modern draughts program can easily get the better of it. I'm sure that Bert, Feike, Gerard, and a few others also have programs that can make truus look somewhat overmatched.

-- Ed

[wellnesswrotter]

In the human world Truus is still considerd to be a very good program (against and analysing humans). It is a good test-case!