Discussion related to cado-nfs

[Greg Marks <marks@member.ams.org>]

Dear Pierrick, 

Thank you for your help.  You're quite right: I had forgotten to change
the definition of tasks.execpath in my params file to my alternative
build directory, created when I compiled the git version.

Downloading a new git version today, recompiling from scratch, making
sure to use the correct execpath, setting tasks.sieve.freerel.threads=8,
I now get this error:

   Error:Generate Free Relations: Program run on server failed with exit code 
1
   Error:Generate Free Relations: Command line was: 
/usr/local/cado-nfs-build-git/sieve/freerel -poly 
/tmp/cado.zCGBRfYgj9/c184.polyselect2.poly -renumber 
/tmp/cado.zCGBRfYgj9/c184.freerel.renumber.gz -lpbr 32 -lpba 32 -out 
/tmp/cado.zCGBRfYgj9/c184.freerel.freerel.gz > 
/tmp/cado.zCGBRfYgj9/c184.freerel.freerel.stdout.1 2> 
/tmp/cado.zCGBRfYgj9/c184.freerel.freerel.stderr.1
   Error:Generate Free Relations: Stderr output follows (stored in file 
/tmp/cado.zCGBRfYgj9/c184.freerel.freerel.stderr.1):
   b'# Warning: parameter verbose_flags is checked by this program but is 
undocumented.\nError, missing -lpbr or -lpba command line argument\nUsage: 
/usr/local/cado-nfs-build-git/sieve/freerel <parameters>\nThe available 
parameters are the following:\n    -poly      input polynomial file\n    
-renumber  output file for renumbering table\n    -out       output file for 
free relations\n    -lpb0      large prime bound on side 0\n    -lpb1      
large prime bound on side 1\n    -pmin      do not create freerel below this 
bound\n    -pmax      do not create freerel beyond this bound\n    -badideals 
file describing bad ideals (for DL)\n    -addfullcol (switch) add a column of 
1 in the matrix (for DL)\n    -t         number of threads\n'
   Traceback (most recent call last):
     File "/usr/local/cado-nfs-2.1.1/scripts/cadofactor/cadofactor.py", line 
72, in <module>
       factors = factorjob.run()
     File "/usr/local/cado-nfs-2.1.1/scripts/cadofactor/cadotask.py", line 
4720, in run
       last_status, last_task = self.run_next_task()
     File "/usr/local/cado-nfs-2.1.1/scripts/cadofactor/cadotask.py", line 
4788, in run_next_task
       return [task.run(), task.title]
     File "/usr/local/cado-nfs-2.1.1/scripts/cadofactor/cadotask.py", line 
2367, in run
       raise Exception("Program failed")
   Exception: Program failed

This is running with Linux Centos 6 x86_64.  Reverting to cado-nfs-2.1.1,
I have no problems.

Sincerely,
Greg

    ------------------------------------------------
   | Greg Marks                                     |
   | Department of Mathematics and Computer Science |
   | St. Louis University                           |
   | St. Louis, MO 63103-2007                       |
   | U.S.A.                                         |
   |                                                |
   | Phone: (314)977-7206        Fax: (314)977-1452 |
   | PGP encryption public key ID: 0x53F269E8       |
   | Web: http://gmarks.org                         |
    ------------------------------------------------

Message from Pierrick Gaudry <pierrick.gaudry@loria.fr>
of August 19, 2015, 08:40:04 +0200, follows:
> It seems that you used a new cadofactor script with an old makefb binary.
> I suggest that you recompile everything from scratch with the new
> downloaded version.
> 
> Also, there are currently many changes occurring in the git version,
> sometimes breaking everything. It is highly recommended to check on
>   https://ci.inria.fr/cado/
> to see if there are failures with the current git version for your
> architecture / OS. For instance, at the time of writing this message, it
> is a very bad idea to git pull if you have a Mac.
> 
> Pierrick
> 
> 
> On Tue, Aug 18, 2015 at 11:57:50PM -0500, Greg Marks wrote:
> > Dear Paul,
> > 
> > I'm encountering into a problem with the git version, downloaded today
> > from here:
> > 
> >     git clone 
> > https://scm.gforge.inria.fr/anonscm/git/cado-nfs/cado-nfs.git 
> > 
> > Running the program with the same parameter settings as before, it
> > terminates with this error:
> >    
> >    ...
> >    Info:Generate Factor Base: Starting
> >    Warning:Command: Process with PID 9943 finished with return code 1
> >    Error:Generate Factor Base: Program run on server failed with exit 
> > code 1
> >    Error:Generate Factor Base: Command line was: 
> > /usr/local/cado-nfs-build/sieve/makefb -poly 
> > /tmp/cado.zCGBRfYgj9/c184.polyselect2.poly -lim 80000000 -maxbits 14 -out 
> > /tmp/cado.zCGBRfYgj9/c184.factorbase.roots.gz -t 8 > 
> > /tmp/cado.zCGBRfYgj9/c184.factorbase.makefb.stdout.1 2> 
> > /tmp/cado.zCGBRfYgj9/c184.factorbase.makefb.stderr.1
> >    Error:Generate Factor Base: Stderr output follows (stored in file 
> > /tmp/cado.zCGBRfYgj9/c184.factorbase.makefb.stderr.1):
> >    b'Error: parameter -alim is mandatory\nUsage: 
> > /usr/local/cado-nfs-build/sieve/makefb <parameters>\nThe available 
> > parameters are the following:\n    -poly      polynomial file\n    -alim  
> >     factor base bound\n    -maxbits   (optional) maximal number of bits 
> > of powers\n    -out       (optional) name of the output file\n    -side   
> >    (optional) create factor base for given side. Side must be 0 or 1 
> > (default is 1, i.e. algebraic).\n'
> >    Traceback (most recent call last):
> >      File "/usr/local/cado-nfs/scripts/cadofactor/cadofactor.py", line 
> > 81, in <module>
> >        factors = factorjob.run()
> >      File "/usr/local/cado-nfs/scripts/cadofactor/cadotask.py", line 
> > 5251, in run
> >        last_status, last_task = self.run_next_task()
> >      File "/usr/local/cado-nfs/scripts/cadofactor/cadotask.py", line 
> > 5329, in run_next_task
> >        return [task.run(), task.title]
> >      File "/usr/local/cado-nfs/scripts/cadofactor/cadotask.py", line 
> > 2453, in run
> >        raise Exception("Program failed")
> >    Exception: Program failed
> > 
> > I'm using a copy of /usr/local/cado-nfs/params/params.c186 (with extra
> > settings appended) as the argument to cadofactor.py, which includes
> > this definition:
> > 
> > alim = 80000000
> > 
> > Using cado-nfs-2.1.1 with the same params file, there are no problems.
> > 
> > Sincerely,
> > Greg 
> > 
> >     ------------------------------------------------
> >    | Greg Marks                                     |
> >    | Department of Mathematics and Computer Science |
> >    | St. Louis University                           |
> >    | St. Louis, MO 63103-2007                       |
> >    | U.S.A.                                         |
> >    |                                                |
> >    | Phone: (314)977-7206        Fax: (314)977-1452 |
> >    | PGP encryption public key ID: 0x53F269E8       |
> >    | Web: http://gmarks.org                         |
> >     ------------------------------------------------
> > 
> > Message from paul zimmermann <Paul.Zimmermann@inria.fr>
> > of August 17, 2015, 18:14:39 +0200, follows:
> > >        Greg,
> > > 
> > > it seems to work with the git version (just fixed the fact that the 
> > > command
> > > line was not printed in c60.freerel.freerel.stdout.1):
> > > 
> > > zimmerma@tarte:~/svn/cado-nfs$ ./factor.sh 
> > > 90377629292003121684002147101760858109247336549001090677693 
> > > tasks.sieve.freerel.threads=3
> > > ...
> > > zimmerma@tarte:~/svn/cado-nfs$ head -1 
> > > /tmp/cado.hDYwnLCKdk/c60.freerel.freerel.stdout.1 
> > > # (747a3bd+) 
> > > /users/caramel/zimmerma/svn/cado-nfs/build/tarte/sieve/freerel -poly 
> > > /tmp/cado.hDYwnLCKdk/c60.polyselect2.poly -renumber 
> > > /tmp/cado.hDYwnLCKdk/c60.freerel.renumber.gz -lpb0 18 -lpb1 19 -out 
> > > /tmp/cado.hDYwnLCKdk/c60.freerel.freerel.gz -t 3
> > > 
> > > Please can you check again (with the latest git version) and report 
> > > here?
> > > 
> > > Paul
> > > 
> > > > Date: Sat, 8 Aug 2015 14:05:16 -0500
> > > > From: Greg Marks <marks@member.ams.org>
> > > > 
> > > > Dear Paul,
> > > > 
> > > > Thank you for appended message containing the patch.  I did change
> > > > the dup2.c file as indicated in your message below, and recompiled.
> > > > The program has been sieving successfully for perhaps a month since
> > > > then, and I have not encountered this zero-byte problem again.  So 
> > > > this
> > > > probably means that problem is solved.
> > > > 
> > > > Incidentally, in another message you suggested that one could speed up
> > > > the free relation calculation when restarting a computation in a fresh
> > > > working directory with the command:
> > > > 
> > > >    cadofactor.py ... tasks.sieve.freerel.threads=3
> > > > 
> > > > I've tried this a few times, including with other numbers in place of
> > > > 3, but the freerel routine uses only one core no matter what I try.
> > > > Admittedly this isn't that important, since it's only an extra hour
> > > > or two from time to time in a months-long computation.  (There is a
> > > > sort of psychological reason to want to speed this step along: what's
> > > > usually happened is that I've deleted some corrupted sieving file and
> > > > am anxious to see whether las is going to restart without errors.)
> > > > 
> > > > Sincerely,
> > > > Greg 
> > > > 
> > > >     ------------------------------------------------
> > > >    | Greg Marks                                     |
> > > >    | Department of Mathematics and Computer Science |
> > > >    | St. Louis University                           |
> > > >    | St. Louis, MO 63103-2007                       |
> > > >    | U.S.A.                                         |
> > > >    |                                                |
> > > >    | Phone: (314)977-7206        Fax: (314)977-1452 |
> > > >    | PGP encryption public key ID: 0x53F269E8       |
> > > >    | Web: http://gmarks.org                         |
> > > >     ------------------------------------------------
> > > > 
> > > > Message from paul zimmermann <Paul.Zimmermann@inria.fr>
> > > > of June 29, 2015, 18:03:18 +0200, follows:
> > > > > > The program now terminates with the error message appended at the 
> > > > > > bottom
> > > > > > of this message.  The file 
> > > > > > $WORKDIR/c184.duplicates1//0/dup1.1.0000.gz
> > > > > > mentioned in the stderr below turns out to be a 20 byte file, 
> > > > > > which,
> > > > > > when gunzipped, is a zero byte file.
> > > > > 
> > > > > it is a bug that a zero byte file is not accepted in dup2. I fixed 
> > > > > this in
> > > > > commit 73aa354 (review welcome).
> > > > > 
> > > > > Greg, please could you try again with this change?
> > > > > 
> > > > > --- a/filter/dup2.c
> > > > > +++ b/filter/dup2.c
> > > > > @@ -432,11 +432,18 @@ int check_whether_file_is_renumbered(const 
> > > > > char * filename, unsigned int npoly)
> > > > >      unsigned int count = 0;
> > > > >      char s[1024];
> > > > >      FILE *f_tmp = fopen_maybe_compressed (filename, "rb");
> > > > > +
> > > > >      if (!f_tmp) {
> > > > >          fprintf(stderr, "%s: %s\n", filename, strerror(errno));
> > > > >          abort();
> > > > >      }
> > > > >  
> > > > > +    if (feof (f_tmp)) /* file is empty */
> > > > > +      {
> > > > > +        fclose_maybe_compressed (f_tmp, filename);
> > > > > +        return 1; /* an empty file might be considered as 
> > > > > renumbered */
> > > > > +      }
> > > > > +
> > > > >      /* Look for first non-comment line */
> > > > >      while (1) {
> > > > >        char *ret = fgets (s, 1024, f_tmp);
> > > > > 
> > > > > Paul
> > > > 
> > > > [2:application/pgp-signature Show Save:signature.asc (819B)]
> > > > 
> 
> 
> 
> > _______________________________________________
> > Cado-nfs-discuss mailing list
> > Cado-nfs-discuss@lists.gforge.inria.fr
> > http://lists.gforge.inria.fr/mailman/listinfo/cado-nfs-discuss

Attachment: signature.asc
Description: Digital signature