Discussion related to cado-nfs

[ROBERT DUCHATEAU <rjd1212@cox.net>]

A couple of issues here regarding factoring C180

1)  About 2 weeks ago, there was a problem in the MERGE portion of the cado-nfs-2.3.0 program where it would just freeze and not run, & then gradually gobbling up all of 64GB of memory in about 3 hours time, where everything freezes up & then I am forced to reboot. ( I now have installed 128GB RAM) The log files that I viewed simply showed no error messages, only that the MERGE portion had been activated, that's it. A mystery indeed. Unfortunately, I don't have possession of that log file for evaluation so I simply decided on starting from scratch by deleting ALL previous work files & compiling the GIT development version cad-nfs-23930BB in order to obtain a more favorable result. This time, the sieve portion ran much more quickly, only taking 13 days on a 10 computer network running 88 threads. So far so good.

2) However, during the sieving process, I noticed something peculiar in the log files - a possible error message perhaps? Or just a harmless anomaly? Here is the relevant portion of that log file:

******************************************************************************************************

PID65537 2018-03-06 15:58:07,045 Debug:Lattice Sieving: Newly arrived stats: {'stats_avg_J': '16650.0 232', 'stats_total_cpu_time': '3943.97'}
PID65537 2018-03-06 15:58:07,045 Debug:Lattice Sieving: Combined stats: {'stats_avg_J': '16511.9229346035 3623481', 'stats_max_bucket_fill': '0.604677', 'stats_total_cpu_time': '61764456.819999844'}
PID65537 2018-03-06 15:58:07,045 Info:Lattice Sieving: Found 14396 relations in '/opt/c180.upload/c180.110845000-110850000.9rrm0jzg.gz', total is now 252010702/260721300
PID65537 2018-03-06 15:58:07,045 Info:Lattice Sieving: Marking workunit c180_sieving_110845000-110850000 as ok (96.7% => ETA Wed Mar 7 01:56:01 2018)
PID65537 2018-03-06 15:58:57,603 Debug:HTTP server: 192.168.137.8 "POST /cgi-bin/upload.py HTTP/1.1" 200 -
PID65537 2018-03-06 15:58:57,603 Debug:HTTP server: 192.168.137.8 Translated path cgi-bin/upload.py to /home/rjd/cado-nfs-23930bb/scripts/cadofactor/upload.py
PID65537 2018-03-06 15:58:57,674 Info:HTTP server: 192.168.137.8 Sending workunit c180_sieving_111010000-111015000 to client home101.d9f57958
PID65537 2018-03-06 15:58:57,675 Debug:HTTP server: 192.168.137.8 "GET /cgi-bin/getwu?clientid=home101.d9f57958 HTTP/1.1" 200 -
PID65537 2018-03-06 15:58:58,117 Info:Lattice Sieving: Adding workunit c180_sieving_111060000-111065000 to database
PID65537 2018-03-06 15:58:58,134 Debug:Lattice Sieving: stderr is: b"# redoing q=110930009, rho=38552643 because buckets are full\n# Fullest level-1s bucket #5208, wrote 504/384\n# Maybe you have too many threads compared to the size of the factor bases.\n# Please try less threads, or a larger -bkmult parameter (at some cost!).\n# The code will now try to adapt by allocating more memory for buckets.\n# redoing q=110930009, rho=38552643 because buckets are full\n# Fullest level-1s bucket #68, wrote 620/544\n# Maybe you have too many threads compared to the size of the factor bases.\n# Please try less threads, or a larger -bkmult parameter (at some cost!).\n# The code will now try to adapt by allocating more memory for buckets.\n# redoing q=110930159, rho=81102186 because buckets are full\n# Fullest level-1s bucket #2168, wrote 768/672\n# Maybe you have too many threads compared to the size of the factor bases.\n# Please try less threads, or a larger -bkmult parameter (at some cost!).\n# The code will now try to adapt by allocating more memory for buckets.\n# redoing q=110930753, rho=86948833 because buckets are full\n# Fullest level-1s bucket #4811, wrote 879/848\n# Maybe you have too many threads compared to the size of the factor bases.\n# Please try less threads, or a larger -bkmult parameter (at some cost!).\n# The code will now try to adapt by allocating more memory for buckets.\n# Average J=16622 for 258 special-q's, max bucket fill -bkmult 1,1s:2.59446\n# Discarded 0 special-q's out of 258 pushed\n# Total cpu time 4789.99s [norm 0.41+3.6, sieving 2087.9 (1149.5 + 39.8 + 898.6), factor 2698.1 (2630.4 + 67.7)]\n# Total elapsed time 988.74s, per special-q 3.83233s, per relation 0.0620562s\n# PeakMemusage (MB) = 13991 \n# Total 15933 reports [0.301s/r, 61.8r/sq]\n"

***************************************************************************************************

This message repeats itself on & on throughout the sieving process.

Question: Is this message a warning that the factorization may fail at some point? A possible bug in the coding perhaps? Or perhaps an adjustment is in order in the C180 parameter file?

Incidentally, I have made a modification per suggestions in the log file pertaining to the "tasks.sieve.bkmult = 1.1" command. That change is in the C180 parameter file which is also in the c190 parameter file in the GIT version.

I really don't know if this change will have any effect on the outcome of the program, since the same stderr message keeps popping up over & over again. Any ideas on what's going on?

Regards,

RJD