Skip to Content.
Sympa Menu

coq-club - Re: [Coq-Club] sound incremental builds

coq-club AT inria.fr

Subject: The Coq mailing list

List archive

Re: [Coq-Club] sound incremental builds


Chronological Thread 
  • From: Jason Gross <jasongross9 AT gmail.com>
  • To: coq-club <coq-club AT inria.fr>
  • Subject: Re: [Coq-Club] sound incremental builds
  • Date: Tue, 18 Aug 2020 02:44:04 -0400
  • Authentication-results: mail2-smtp-roc.national.inria.fr; spf=None smtp.pra=jasongross9 AT gmail.com; spf=Pass smtp.mailfrom=jasongross9 AT gmail.com; spf=None smtp.helo=postmaster AT mail-ed1-f52.google.com
  • Ironport-phdr: 9a23:gTWI6haaR0D+TcoBnHr9rPb/LSx+4OfEezUN459isYplN5qZr8u7bnLW6fgltlLVR4KTs6sC17OI9fm4BCdZucrJmUtBWaQEbwUCh8QSkl5oK+++Imq/EsTXaTcnFt9JTl5v8iLzG0FUHMHjew+a+SXqvnYdFRrlKAV6OPn+FJLMgMSrzeCy/IDYbxlViDanbr5+MRS7oR/MusUKgYZuJbs9xgfGr3BVZ+lY2GRkKE6JkR3h/Mmw5plj8ypRu/Il6cFNVLjxcro7Q7JFEjkoKng568L3uxbNSwuP/WYcXX4NkhVUGQjF7Qr1UYn3vyDnq+dywiiaPcnxTbApRTSv6rpgRRH0hCsbMTMy7WfagdFygq1GuhKsvxNww4DWb4+VOvRwfb7Tc80GSmdaRMldSzZMD5mgY4cTDecMO/tToYnnp1sJqBuzHQ2iC/n0yj9PgH/9wKw00/4mEQHdxwwrAtUDsHbOo9XuM6cSV++1zLPPzTXYdPNb3Szw6IfNch87oPGMWah8ftbWyUkqDg7IiEibpoP5MT2PzOsNr3Sb4PR6VeKpk2Mqqx18riSzyskiiITEm4EYx1TK+yt33Ys4K9K1RVJlbNO5DpdduT+WOoh4TM8+QWxlpCU3x7IGt5KnciUG1JopyhjCYPKJdIiI5wjsVOeXITpgi3Jlea6/hxav8Ue70OHzSs600FNSoiVZldnMrHYN2ALX6siDUPdy4Fqu2SuJ2gvO6e9EOVg5mbTHJ5Ml2LI9lZoevV7dEiPqm0j6lq+belsi9+O18eroeK/mqYWZN4JsigHxLKAumsunDOQ9KAcOXmyb9f2i27L+4EH1WbtKg/0onqXDv5DaIsMbpqG9AwBLyIos9xG/DzK+3NQZm3kIMk5FdQqZg4T1P1zCOvP1APelj1iyjDtmxOrKM73/DpnVK3jMirbhfbJz605GzwozyMhS55BOBbEaJ/LzXEDxtMbfDh8iKAy5x/3qCNp41owEWGKPBrWVP7/VsV+N/u4vOfWDZJcJuDbhLPgo/+LhjXggmVMEYaap2YYXZ2ujE/R9I0SZZGLsjc0bHWcLuAo+Vu3qh0eYXT5dfXbhF547szo8EcetCZrJDtSmh6XE1yOmFLVXYHpHAxaCCyG7WZ+DXqIuYTmVJIdOiDsfTvD1SYY60hehrgjh0OtPIe/d+ylevpXmgosmr9bPnA0/oGQnR/+W1HuAGjktwjE4AgQu1aU6mnRTj0+Z2PEh0fNdHN1XofhOV1VibM+O/6lBE9n3Hzn5UJKMQVeiTM+hBGhoHN00yt4KJU16Hof7102R72+RG7YQ0oezKtk0/6bbhSWjIs98zzPL2PBkgQR8BMRIMmKii+h08A2BX4M=

This will be fixed in Coq 8.13, and is already fixed in master; see https://github.com/coq/coq/pull/12427

On Tue, Aug 18, 2020, 02:35 Abhishek Anand <abhishek.anand.iitg AT gmail.com> wrote:
Thanks. I found this non-transient timestamp-related issue, where if a file older than the last coqdep invocation is added to _CoqProject, coqdep is not reinvoked, leading to unsould incremental builds:

[abhishek@optiplex temp]$ ls -lrt
total 8
-rw-r--r-- 1 abhishek abhishek  0 Aug 17 22:32 A.v
-rw-r--r-- 1 abhishek abhishek 14 Aug 17 23:26 _CoqProject
-rw-r--r-- 1 abhishek abhishek 23 Aug 17 23:27 B.v
[abhishek@optiplex temp]$ cat _CoqProject  
-R . test
A.v
[abhishek@optiplex temp]$ coq_makefile -f _CoqProject -o Makefile
[abhishek@optiplex temp]$ make
COQDEP VFILES
COQC A.v
[abhishek@optiplex temp]$ echo "B.v" >> _CoqProject  
[abhishek@optiplex temp]$ cat B.v
Require Import test.A.
[abhishek@optiplex temp]$ rm A.vo
[abhishek@optiplex temp]$ make B.vo
coq_makefile -f _CoqProject -o Makefile
COQC B.v
File "./B.v", line 1, characters 15-21:
Error: Unable to locate library test.A.

make: *** [Makefile:678: B.vo] Error 1

[abhishek@optiplex temp]$ touch B.v
[abhishek@optiplex temp]$ make B.vo
COQDEP VFILES
COQC A.v
COQC B.v

A workaround is to also delete .Makefile.d (coqdep output) whenever _CoqProject is updated.

On Sat, Jul 18, 2020 at 5:32 PM David Holland <dholland-coq AT netbsd.org> wrote:
On Fri, Jul 17, 2020 at 06:24:31PM -0700, Abhishek Anand wrote:
 > I want to avoid ever having to do clean builds (`make clean`) and still
 > ensure soundness of the build.
 > [...]
 > Below is an approach that I think works. I'd appreciate counterexamples or
 > arguments why this is correct:
 > 1) delete every .vo, .vio, .vos, .glob file that does not correspond to a
 > .v file in X/_CoqProject.
 > 2) reinvoke coq_makefile on X/_CoqProject to regenerate X/Makefile
 > 3) `make`.

Don't forget to also delete leftover .foo.aux files.

That said, there are always cases where you have to make clean,
because make relies on timestamps to detect when files have been
changed and timestamps can get screwed up. For example, if you
accidentally set your clock forward a week and work for a while before
fixing it, you'll likely be left with build products dated in the
future that won't be rebuilt properly.

Timestamps also don't (normally) cover all the possible things that
might change your build; e.g. when you update Coq you need to make
clean, and you generally want or need to if you change build settings.

And if the storage you're working from runs over a network (including
most cases of being shared inside and outside a VM) you can get
various problems arising from clock skew, and if you access such
storage from two or more places, sometimes cache (in)consistency as
well. (The latter is particularly bad with NFS because there are no
consistency guarantees at all, just some ad hoc timeouts, and
timestamp changes don't necessarily propagate at the same time as file
contents changes.)

Other than things like that though... make is supposed to give you
consistent builds, and coq_makefile is supposed to produce correct
makefiles, so if it doesn't work it's a bug in (most likely)
coq_makefile or (quite unlikely) make or the OS.

--
David A. Holland
dholland AT netbsd.org



Archive powered by MHonArc 2.6.19+.

Top of Page