8 pieces endgame database

[TAILLE]

Seems to be the same as my numbers

Michel

seems ?????

[TAILLE]

Hi,

Damy continue to progress in the 8 db generation.
Now Damy has generated :
the complete 4x4 db
the 5 pieces(men or kings) against 3K db
all counters are identical to Michel's and Ed's

Michel : have you finished the 5x3 db generation ?

[MichelG]

Michel : have you finished the 5x3 db generation ?

I had a small delay because i ran out of disk space and had to reprogram some things to reduce the amount of temporary space it uses. But 5x3 is running full speed now, and is at about 70-75% done. I guess it is 2 more weeks and then onto 6x2.

Michel

[Rein Halbersma]

Michel : have you finished the 5x3 db generation ?

I had a small delay because i ran out of disk space and had to reprogram some things to reduce the amount of temporary space it uses. But 5x3 is running full speed now, and is at about 70-75% done. I guess it is 2 more weeks and then onto 6x2.

Michel

How much disk space are you going to use? IIRC, the Dragon version that is on your website produces a lot of intermediate files and has a much larger total disk size compared to e.g. the Flits/Truus dbs. Did you change your format for the 8 pc dbs?

Rein

[MichelG]

Dragon has several versions of each database;
- the uncompressed data (2 bits/position, including captures)
- lightly compressed databases which includes all captures (about 1 bit/position)
- highly compressed databases which don't store captures (less then 1 bit/position). These are used during gameplay.

The old version stored all of the uncompressed databases + the compressed databases with captures. The uncompressed are used for performance when possible, the compressed are used to save ram when needed. As you can imagine this uses a huge amount of storage. (i got a 2 TB disk dedicated to dragon)

The current version only stores the compressed databases including captures and uncompresses them on the fly when possible.

When all computation is done, i will compresses a selection of them (i don't care about positions with 2 or more kings in them on a side, they are almost draw by definition), and it will end up with a couple of tens of gigabytes.

Another idea to reduce the amount of temporary space, is to remove the databases that you don't need anymore. For example, once you compute 1304 and 0413, you can just remove 0404, because there are no conversions into that anymore.

Michel

[BertTuyt]

Michel, just for my understanding.

When you generate the DB you have an uncompressed version in memory which you compress at the end?
And next to that do you complete load compressed sub-databases during the DB-built,or do you use (as I have nowadays) a caching mechanism based on (for example) 4 KByte blocks?

Bert

[MichelG]

Michel, just for my understanding.

When you generate the DB you have an uncompressed version in memory which you compress at the end?
And next to that do you complete load compressed sub-databases during the DB-built,or do you use (as I have nowadays) a caching mechanism based on (for example) 4 KByte blocks?

Bert

Well, the database that is being constructed is always uncompressed. But most of the RAM is used on the depended databases. These can either be uncompressed or compressed, depending on whether or not they fit in RAM.

Dragon does not have any block based cache mechanism. It just loads and unloads the full databases. But in retrospect, i think a caching mechanism would make memory management a bit more easy to program.

Michel

[MichelG]

Both 4x4 and 5x3 have finally been build

Last week the memory chips in my computer broke down, but i took out the faulty chips and let it continue with half the memory.

The entire computation took a bit more than 3 months on 6 cores. About 2.5 months was spend on computing, and about 3 weeks spend on delays caused by adjustments to the program to deal with memory failures and lack of disk space.

The counts are identical to Ed's numbers, thus confirming his work. Congratulations .

Dragon is now working on computing 6x2. Am i right to assume you did not compute these databases, Ed?

Michel

[Rein Halbersma]

Both 4x4 and 5x3 have finally been build

Last week the memory chips in my computer broke down, but i took out the faulty chips and let it continue with half the memory.

The entire computation took a bit more than 3 months on 6 cores. About 2.5 months was spend on computing, and about 3 weeks spend on delays caused by adjustments to the program to deal with memory failures and lack of disk space.

The counts are identical to Ed's numbers, thus confirming his work. Congratulations .

Dragon is now working on computing 6x2. Am i right to assume you did not compute these databases, Ed?

Michel

Hi Michel,

Congratulations on this impressive feat!

Could you also update your statistics web page for these endgames like you have done for the 2-7 piece databases? In particular, in light of the thread on unique losses, could you also update the 6 piece db statistics to also include non-capture positions?

Rein

[Ed Gilbert]

Both 4x4 and 5x3 have finally been build :-)

Last week the memory chips in my computer broke down, but i took out the faulty chips and let it continue with half the memory.

The entire computation took a bit more than 3 months on 6 cores. About 2.5 months was spend on computing, and about 3 weeks spend on delays caused by adjustments to the program to deal with memory failures and lack of disk space.

The counts are identical to Ed's numbers, thus confirming his work. Congratulations :-).

Dragon is now working on computing 6x2. Am i right to assume you did not compute these databases, Ed?

Michel

Congratulations, and nice to know that our counts agree.

No, I did not build the 6x2 subset.

-- Ed

[Rein Halbersma]

Both 4x4 and 5x3 have finally been build

Last week the memory chips in my computer broke down, but i took out the faulty chips and let it continue with half the memory.

The entire computation took a bit more than 3 months on 6 cores. About 2.5 months was spend on computing, and about 3 weeks spend on delays caused by adjustments to the program to deal with memory failures and lack of disk space.

The counts are identical to Ed's numbers, thus confirming his work. Congratulations .

Dragon is now working on computing 6x2. Am i right to assume you did not compute these databases, Ed?

Michel

Congratulations, and nice to know that our counts agree.

No, I did not build the 6x2 subset.

-- Ed

Ed,

Now that there has been a "second Moon landing", do you have any plans to push the state of the art any further? Full 9 piece dbs? Or more interesting for practical playing strenght: 5m vs 5m incomplete dbs? Is it feasible within a year's computation, how much disk space, etc?

Rein

[Ed Gilbert]

Now that there has been a "second Moon landing", do you have any plans to push the state of the art any further? Full 9 piece dbs? Or more interesting for practical playing strenght: 5m vs 5m incomplete dbs? Is it feasible within a year's computation, how much disk space, etc?

I might be able to build 5m vs. 5m but it would be too large to use effectively. The 4m vs. 4m is 0.7GB, 5m vs. 4m is 20GB, so you can estimate how large 5m vs. 5m might be. I already have 5m vs. 3m 1k and 4m 1k vs. 4m but I don't use them as they are so large. Maybe when typical desktop PCs have at least 64GB of ram we can start to think about using such large dbs.

-- Ed

[BertTuyt]

Michel, well done.

When do you post the 8p DB stats on your website?
I'm looking forward to it.

Bert

[TAILLE]

Both 4x4 and 5x3 have finally been build

Last week the memory chips in my computer broke down, but i took out the faulty chips and let it continue with half the memory.

The entire computation took a bit more than 3 months on 6 cores. About 2.5 months was spend on computing, and about 3 weeks spend on delays caused by adjustments to the program to deal with memory failures and lack of disk space.

The counts are identical to Ed's numbers, thus confirming his work. Congratulations .

Dragon is now working on computing 6x2. Am i right to assume you did not compute these databases, Ed?

Michel

Congratulation Michel; I am far behind you :
4x4 = 100%
5x3 = 40%

[BertTuyt]

Michel,

maybe this question was already covered or answered.
Ed uses multiple instances of the DB-gen program to generate all DB's.
In your case did you use a parallel version of your Generator , or did you had a similar approach as Ed?
If you used a parallel DB-Gen, how does it scale with the number of cores.
As the core number will explode the next years, this would be good news for all of us .

Bert