Internet engine matches

[Ed Gilbert]

I think the problem during tournaments is not. Tournaments are played on two separate computers. Pondering doesn't matter.

In that case, you can do the same with DXP, use two separate computers. DXP uses LAN, it does not require the programs to be on the same computer.

[Krzysztof Grzelak]

I don't know Ed how it's done. I have never done this before.

[Krzysztof Grzelak]

Subject of the match at the moment not current.

[Ed Gilbert]

I don't know Ed how it's done. I have never done this before.

It is basically the same as when running on one computer, except that the Initiator gives the IP address of the Follower computer instead of the loopback address 127.0.0.1. It is also necessary to tell your router to forward the DXP port (27531).

[Ed Gilbert]

I just tested it here. I will describe how to set it up.

1) Find the IP address of the computer that will be running the DXP Follower program. One way to do this is to open up a command prompt on the Follower computer and type

ipconfig

Look for the IPv4 address under your active connection.

2) Open a web browser and login to you router. If you've never done this, you may find the login info on the label on your router. Look for something like "Administration URL" and "Administration password". If it isn't there, or it doesn't seem to be correct, then ask you internet provider how to login to your router.

3) Once logged into the router, look for something called "port forwarding". It might be in a section for Firewall settings, or you might have to hunt around for it, but it shouldn't be difficult to find, as it is a common thing to have to configure in a router. Create a new port forwarding rule. The rule may need a hostname or host IP, and you enter the host info of the Follower computer. The rule will need a port number, which you set to 27531. The rule will probably need to know if it is for udp, tcip, or both. Select both. Save the settings, and you're done. Some routers might need to be rebooted after a setting change, so you can try that if forwarding doesn't seem to work right away.

Now you can start the Follower engine, set it to "wait for connections", then start the Initialtor engine, and set it to connect to the Follower's IP address, etc.

[Krzysztof Grzelak]

We'll slow down Ed a little. Ed, I don't have any router. I have two computers not connected to the internet. I will connect them with a cable LAN.

[jj]

Krzysztof's machine has 16 cores/32 threads, IIRC. So theoretically 16 + 8 = 24 threads is possible. However, a hyper-thread is not the same as a real thread (core), so when pondering is used there can still be interference.

I agree with Ed that switching off pondering when using two programs on one machine is the best solution.

Also, I don't trust results using hyper-threading and I suggest switching it off (in the BIOS settings). In my opinion, hyper-threading is only fine if you have independent threads and that is not the case here.

I would like to add that if two programs support parallel search they should use the same number of threads.

Jan-Jaap

[Ed Gilbert]

Krzysztof's machine has 16 cores/32 threads, IIRC. So theoretically 16 + 8 = 24 threads is possible. However, a hyper-thread is not the same as a real thread (core), so when pondering is used there can still be interference.

24 threads are always possible of course, even on a single-core machine, but with terrible performance. The issue is how many real cores is each program getting, and how many hyper-threads. Windows will divvy them up somehow, in a way that may be more advantageous to one program than another.

Also, I don't trust results using hyper-threading and I suggest switching it off (in the BIOS settings). In my opinion, hyper-threading is only fine if you have independent threads and that is not the case here.

I find hyper-threading to be beneficial for many things, and I never turn it off, even when I am running many engine matches in parallel to test something. I have 8 cores, and I never run more than 8 engine matches in parallel, but it seems like those 8 engine matches are all getting real cores, not hyper-threads, based on the search speeds that I see.
On my CPU a hyper-thread has about 60% of the performance of a real core in an engine search. This makes it useful for things like building databases. I typically run 12 or 14 parallel threads in those cases, and get more work done than just using 8 threads.

I would like to add that if two programs support parallel search they should use the same number of threads.

I guess it depends on what you're trying to measure. If your're trying to measure which program has the better search, then maybe you want to use the same number of search threads, the same size egdb, turn off opening books, etc. I get the impression that Krzysztof is trying to see which program can do best when it's used to its fullest potential, so in that case if one program can use 16 threads and another can only use 8, that's the right way to configure them.

-- Ed

[jj]

Okay, so it is probably not necessary to switch off hyper-threading. In engine matches either switch off pondering or use the rule of thumb that the total number of simultaneous threads never exceeds the number of cores.

[Jelle Wiersma]

Krzysztof, thank you for running the match.

I would like to point out what looks like a problem with your match configuration. You have sjende blyn set to use 16 search threads and pondering on, and kingsrow to use 8 search threads and pondering on. That means that both program are searching at the same time using a total of 24 threads. But your CPU only has 16 cores. If I have all these details correct, it means the two programs are interfering with each other, unless they are running on separate computers. The simple solution is to turn pondering off when both programs are running on the same computer.

edit: I will add that even if the sum of search threads of the two programs was no more than 16, the two programs are probably still interfering with each other to some extent, as they share access to some cache blocks, hard drive, and perhaps other resources. My advice is to never use pondering when both programs are running on one computer.

-- Ed

Indeed that configuration is wrong.
But switching off pondering, and hence the search threads, alone does not solve it.
There can also be memory conflicts because of the memory used by the two programs for the transposition table and endgame database;
Windows will then start swapping memory blocks from RAM to- and from the disk.
Besides the search threads, there can also be thread(s) to load the endgame database into memory, especially for 8-piece database.
These threads also need to be postponed when it is not the program's turn to do a move,
otherwise it will interfere with the other program.
Everything need to be set/implement such that it prevents interference of the two programs.
That's why I proposed to Krzysztof to use two AMD Ryzen, as in the tournament, seemed to be the most simple.

Jelle

[Krzysztof Grzelak]

I just tested it here. I will describe how to set it up.

1) Find the IP address of the computer that will be running the DXP Follower program. One way to do this is to open up a command prompt on the Follower computer and type

ipconfig

Look for the IPv4 address under your active connection.

2) Open a web browser and login to you router. If you've never done this, you may find the login info on the label on your router. Look for something like "Administration URL" and "Administration password". If it isn't there, or it doesn't seem to be correct, then ask you internet provider how to login to your router.

3) Once logged into the router, look for something called "port forwarding". It might be in a section for Firewall settings, or you might have to hunt around for it, but it shouldn't be difficult to find, as it is a common thing to have to configure in a router. Create a new port forwarding rule. The rule may need a hostname or host IP, and you enter the host info of the Follower computer. The rule will need a port number, which you set to 27531. The rule will probably need to know if it is for udp, tcip, or both. Select both. Save the settings, and you're done. Some routers might need to be rebooted after a setting change, so you can try that if forwarding doesn't seem to work right away.

Now you can start the Follower engine, set it to "wait for connections", then start the Initialtor engine, and set it to connect to the Follower's IP address, etc.

Ed computer is not connected to the internet. Connecting two computers is as follows:

COM 1 - cable ethernet - router (without internet) - cable ethernet - COM 2

Computers see each other. Unfortunately, the programs cannot be seen.

[Ed Gilbert]

Unfortunately, the programs cannot be seen.

You don't need internet, just the router. Did you do the port forwarding?

[Krzysztof Grzelak]

You don't need internet, just the router. Did you do the port forwarding?

Yes. Unfortunately, something is not working. My router is TP link TD-8840T. I'd give it peace.I shone a light three days and it's no use there is no.

[Krzysztof Grzelak]

Ed quite different can be done. You don't have to port forwarding.

[jj]

Ed quite different can be done. You don't have to port forwarding.

Krzysztof, does this mean you have it working now?