Discussion related to cado-nfs

[Alexander Kruppa <alexander.kruppa@inria.fr>]

----- Original Message -----
> From: "Sam Stoelinga" <sammiestoel@gmail.com>
> To: "paul leyland" <paul.leyland@gmail.com>
> Cc: cado-nfs-discuss@lists.gforge.inria.fr
> Sent: Saturday, 1 March, 2014 12:11:05 PM
> Subject: Re: [Cado-nfs-discuss] Cado-nfs fails on duplicate removal
> 
> Hi Paul,
> 
> I'm running 3.3.4 so is the bug also affecting me? I guess it is unless
> they fixed the bug in python, unlikely.
> 
> Sam

There are no updates on the bug: http://bugs.python.org/issue19003
Thus I'm quite certain that the bug remains in Python 3.3.4. I this case, 
please change in wuclient2.py the list of affected versions from

BUGGY_MIMEENCODER2 = (
    (3,2,4), (3,2,5),
    (3,3,2)
)

to

BUGGY_MIMEENCODER2 = (
    (3,2,4), (3,2,5),
    (3,3,2), (3,3,3), (3,3,4)
)


Alex



> 
> 
> On Sat, Mar 1, 2014 at 6:58 PM, Paul Leyland <paul.leyland@gmail.com> wrote:
> 
> > I had similar problems.  Alex and Paul (the other one) helped me by
> > pointing to a set of bugs in Python's MIME encoder.
> >
> > The workaround for me was to follow Alex' advice:
> >
> > > There is an entry on http://cado-nfs.gforge.inria.fr/ under Known Bugs;
> > > do you happen to use Python 3.3.3 on the slaves? If yes, does adding
> > > (3,3,3) (without quotes) to the BUGGY_MIMEENCODER2 list in wuclient2.py
> > > fix it?
> >
> > YMMV.
> >
> >
> > Paul (this one)
> >
> >
> > On Sat, 2014-03-01 at 18:48 +0800, Sam Stoelinga wrote:
> > > Hi Pierrick,
> > >
> > >
> > > Thanks for the fast reply!
> > >
> > >
> > > 2.0 Version. Actually there were like 100+ files unrecoverable it
> > > seems. I tried similar commands.
> > > I used the Archlinux cado-nfs package from the aur.
> > >
> > >
> > > I think the disk should be fine on that machine, although I can;t
> > > promise that. I had the same problem on Amazon EC2 tried it 2 times.
> > >
> > >
> > > This is the folder containing al related files:
> > > http://pan.baidu.com/s/1hqgJDGg (Sorry for Chinese mirror)
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Sat, Mar 1, 2014 at 3:40 PM, Pierrick Gaudry
> > > <pierrick.gaudry@loria.fr> wrote:
> > >         Hi,
> > >
> > >         Which version of cado-nfs did you use? The 2.0 or a recent git
> > >         version ?
> > >
> > >         At first sight, it seems that the error comes from a corrupted
> > >         relation
> > >         file. Maybe running
> > >           cd /tmp/cado.r1psyafiQA/c127.upload/
> > >           for i in c127.sieving.*; do gzip -t $i; done
> > >         will reveal a file that is not well formed.
> > >         If you put a tarball of your /tmp/cado.r1psyafiQA/ directory
> > >         somewhere for download, I can have a look.
> > >
> > >         You are not the first one who experienced problem with bad
> > >         relation
> > >         files. But it is hard to tell whether this is a bug in
> > >         Cado-nfs, or a
> > >         problem with disks...
> > >
> > >         Maybe we should add more checks that the files are correctly
> > >         written to
> > >         disk and that the compression did well.
> > >
> > >         Regards,
> > >         Pierrick
> > >
> > >
> > >         On Sat, Mar 01, 2014 at 10:29:43AM +0800, Sam Stoelinga wrote:
> > >         > Hi,
> > >         >
> > >         >
> > >         > I get the following error while trying to factor:
> > >         >
> > >         > gzip: stdin: invalid compressed data--format violated
> > >         > filter_rels_producer_thread: load error (Bad file
> > >         descriptor) from
> > >         >
> > >         > It tries to execute this command it seems:
> > >         > ......... many more filesl_2ed.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.19687000-19690000.yr6eid.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.15931000-15934000.s9b_08.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.20965000-20968000.zo24c6.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.10804000-10807000.jszv9k.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.15352000-15355000.bpjgj5.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.23434000-23437000.u7_2tw.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.22465000-22468000.c21z8k.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.10699000-10702000.gq01be.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.21277000-21280000.o38532.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.15889000-15892000.as2trp.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.10231000-10234000.1sv89b.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.19729000-19732000.li0pv7.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.22060000-22063000.3_pxqo.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.15025000-15028000.sptcun.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.15217000-15220000.s5pszl.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.16585000-16588000.0q5cw7.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.22057000-22060000.efhup3.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.17077000-17080000.7jnsok.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.20683000-20686000.bdzp9e.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.18061000-18064000.4ybs_h.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.20785000-20788000.5ehwy1.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.18913000-18916000.xt3ama.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.15985000-15988000.ypj3cw.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.12790000-12793000.6lb_cc.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.10537000-10540000.3vbz_n.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.21955000-21958000.eyqdof.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.23818000-23821000.mht30j.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.16324000-16327000.4imyz7.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.17479000-17482000.685wuc.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.16504000-16507000.9sosap.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.23845000-23848000.ze28d6.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.21811000-21814000.zwhazw.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.15631000-15634000.x887_u.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.16942000-16945000.6htns3.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.12475000-12478000.aphbum.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.23674000-23677000.9wx6nt.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.21424000-21427000.pfh974.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.21874000-21877000.2imoc1.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.17326000-17329000.q4j94g.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.10894000-10897000.f0nc93.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.17944000-17947000.v9uz6n.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.20047000-20050000.kyuqxn.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.19468000-19471000.on99v0.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.16534000-16537000.g273_e.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.18736000-18739000.wtama2.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.17053000-17056000.eucwjm.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.22261000-22264000.m3mjz8.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.17353000-17356000.krizj8.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.13408000-13411000.paz5qu.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.17134000-17137000.3y1mfj.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.11803000-11806000.wlkoak.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.22570000-22573000.c43yts.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.14566000-14569000.gathsf.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.14575000-14578000.ff1a_4.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.19345000-19348000.gi8953.gz
> > >         >
> > /tmp/cado.r1psyafiQA/c127.upload/c127.sieving.17836000-17839000.f6cc26.gz 
> > |
> > >         > gzip -dc -
> > >         >
> > >         > Any help would be greatly appreciated! I can provide more
> > >         info as needed.
> > >         >
> > >         > Context: I'm using Cado-nfs to decrypt files which were
> > >         encrypted by some
> > >         > malware following this guide:
> > >         >
> > >
> > http://blog.cassidiancybersecurity.com/post/2014/02/Bitcrypt-broken
> > >         > I'm doing this for a friend.
> > >         >
> > >         > Params used:
> > >         > slaves.basepath = /tmp/cado.r1psyafiQA/client/
> > >         > slaves.hostnames = localhost
> > >         > slaves.nrclients = 20
> > >         > slaves.niceness = 10
> > >         > slaves.scriptpath = /usr/libexec/cado-nfs/bin
> > >         > name = c127
> > >         > rlim = 5000000
> > >         > server.address = localhost
> > >         > N =
> > >         >
> > >
> > 6197414403066148465767611830477972406781444626901692539671142017622145058977671120729249030073292775561014115285376040234795763
> > >         > alim = 10000000
> > >         > lpba = 27
> > >         > tasks.merge.coverNmax = 100
> > >         > tasks.execpath = /usr/libexec/cado-nfs/bin
> > >         > tasks.sieve.rels_wanted = 15500000
> > >         > tasks.sieve.mfba = 81
> > >         > tasks.sieve.mfbr = 54
> > >         > tasks.sieve.rlambda = 2.1
> > >         > tasks.sieve.qrange = 3000
> > >         > tasks.sieve.qmin = 10000000
> > >         > tasks.sieve.alambda = 2.8
> > >         > tasks.sieve.I = 12
> > >         > tasks.linalg.bwc.interval = 1000
> > >         > tasks.linalg.bwc.interleaving = 0
> > >         > tasks.linalg.bwc.mn = 64
> > >         > tasks.linalg.bwc.shuffled_product = 1
> > >         > tasks.linalg.characters.nchar = 50
> > >         > tasks.filter.merge.forbw = 3
> > >         > tasks.filter.purge.keep = 160
> > >         > tasks.filter.ratio = 1.1
> > >         > tasks.filter.maxlevel = 20
> > >         > tasks.threads = 1
> > >         > tasks.polyselect.incr = 60
> > >         > tasks.polyselect.maxnorm = 41.7
> > >         > tasks.polyselect.incr = 60
> > >         > tasks.polyselect.maxnorm = 41.7
> > >         > tasks.polyselect.admax = 2000000
> > >         > tasks.polyselect.degree = 5
> > >         > tasks.polyselect.adrange = 20000
> > >         > tasks.polyselect.P = 50000
> > >         > tasks.polyselect.nq = 1000
> > >         > tasks.workdir = /tmp/cado.r1psyafiQA
> > >         > lpbr = 27
> > >
> > >
> > >         > _______________________________________________
> > >         > Cado-nfs-discuss mailing list
> > >         > Cado-nfs-discuss@lists.gforge.inria.fr
> > >         >
> > >
> > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/cado-nfs-discuss
> > >
> > >
> > >
> > > _______________________________________________
> > > Cado-nfs-discuss mailing list
> > > Cado-nfs-discuss@lists.gforge.inria.fr
> > > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/cado-nfs-discuss
> >
> >
> >
> 
> _______________________________________________
> Cado-nfs-discuss mailing list
> Cado-nfs-discuss@lists.gforge.inria.fr
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/cado-nfs-discuss
>