From jasmin.blanchette at gmail.com Mon Mar 19 11:52:36 2012 From: jasmin.blanchette at gmail.com (Jasmin Blanchette) Date: Mon, 19 Mar 2012 12:52:36 +0100 Subject: [metis-users] Parameters and incompleteness Message-ID: Hi Joe, I was wondering whether custom settings of the Metis parameters could lead to incompleteness. More specifically, Isabelle has so far operated with val waiting_params = {symbolsWeight = 1.0, variablesWeight = 0.0, literalsWeight = 0.0, models = []} and I've found examples that seem to loop forever (who knows really), but that are solved very quickly with "variablesWeight = 0.5". Should "variablesWeight" and "literalsWeight" be nonzero? Vampire is not a model of documentation, but they nonetheless documented which values of which options are incomplete [1]. This could be an idea for Metis as well. While I'm mentioning documentation issue, I've been bitten by assuming that ":" and "." (like all the other pretty-printed operators) were uninterpreted -- but there is special support for those in Metis, and incompleteness seem to rear its ugly head if these are not used in the intended way. [1] https://docs.google.com/View?id=dcqg43v9_44d43x8ngj Regards, Jasmin From jasmin.blanchette at gmail.com Tue Mar 20 00:16:48 2012 From: jasmin.blanchette at gmail.com (Jasmin Blanchette) Date: Tue, 20 Mar 2012 01:16:48 +0100 Subject: [metis-users] KBO vs. LPO Message-ID: <2BB9CBEF-48A9-4E2C-BA07-48F602671ECB@gmail.com> Hi Joe, KBO is hard-coded in Metis. Although KBO is generally acknowledged to "order more things" than LPO, there are some advantages to LPO, such as the possibility of orienting map f (Cons x xs) = Cons (f x) (map f xs) (and in general equations where variables occur several times on the right-hand side) left-to-right. Combined with an appropriate precedence function (that reflects the order in which symbols are defined in HOL), this could do wonders. From what I understand, it would be fairly straightforward to adapt Metis's code to support LPO as well -- or am I mistaken? What are your views on this? Would you be ready to consider an eventual patch that adds LPO, or at least the possibility of plugging-in non-KBO term orderings, for integration in the Metis repository? Thanks in advance. Cheers, Jasmin From joe at gilith.com Wed Mar 21 18:26:21 2012 From: joe at gilith.com (Joe Hurd) Date: Wed, 21 Mar 2012 11:26:21 -0700 Subject: [metis-users] Parameters and incompleteness In-Reply-To: References: Message-ID: Hi Jasmine, > I was wondering whether custom settings of the Metis parameters could lead to incompleteness. Yes indeed, I would describe parameter setting as a minefield. > More specifically, Isabelle has so far operated with > > ?val waiting_params = > ? ?{symbolsWeight = 1.0, > ? ? variablesWeight = 0.0, > ? ? literalsWeight = 0.0, > ? ? models = []} > > and I've found examples that seem to loop forever (who knows really), but that are solved very quickly with "variablesWeight = 0.5". Should "variablesWeight" and "literalsWeight" be nonzero? The waiting_params control the order that clauses are removed from the empty set. The way that Metis implements this is that there is a default FIFO-ish weight (the "distance" of a clause from the initial clauses), and the parameters in the waiting_params structure modify the default weight. I *believe* that this procedure will eventually remove every clause from the waiting set, and so is complete (but proving this is rather subtle and depends on the whole loop). > Vampire is not a model of documentation, but they nonetheless documented which values of which options are incomplete [1]. This could be an idea for Metis as well. I think that would be a good idea, but would take significant time to do a good job (so may not happen). > While I'm mentioning documentation issue, I've been bitten by assuming that ":" and "." (like all the other pretty-printed operators) were uninterpreted -- but there is special support for those in Metis, and incompleteness seem to rear its ugly head if these are not used in the intended way. Yes, it would be safest to regard these as reserved words and avoid these. They are interpreted to solve specific problems that came up in practice: "variable : type" terms were unifying with all possible paramodulations "s : type = t : type", but it's complete to skip paramodulations into variables. Metis interprets "variable : type" terms as variables for this purpose to block paramodulation (since ":" is morally just an annotation). "function . argument" terms encode higher order function applications, and Metis interprets them this way for the purpose of weighting clauses using finite models. [Isabelle doesn't use this heuristic, so this shouldn't matter.] Hope that helps to clarify things. Cheers, Joe From jasmin.blanchette at gmail.com Wed Mar 21 22:21:40 2012 From: jasmin.blanchette at gmail.com (Jasmin Blanchette) Date: Wed, 21 Mar 2012 23:21:40 +0100 Subject: [metis-users] Parameters and incompleteness In-Reply-To: References: Message-ID: Hi Joe, > I think that would be a good idea, but would take significant time to > do a good job (so may not happen). Well, at least there's now a mailing list and an archive, so some of the information is now stored somewhere. ;) > "variable : type" terms were unifying with all possible > paramodulations "s : type = t : type", but it's complete to skip > paramodulations into variables. Metis interprets "variable : type" > terms as variables for this purpose to block paramodulation (since ":" > is morally just an annotation). Ah, I see. I also used ": type" for types, which seemed reasonable, but this was done in an unsystematic way (using some clever but legetimate tricks), where paramodulation into "variable : type" appears to be necessary. > "function . argument" terms encode higher order function applications, > and Metis interprets them this way for the purpose of weighting > clauses using finite models. [Isabelle doesn't use this heuristic, so > this shouldn't matter.] OK. We're actually using "." the way it's intended to be used, so this never was a problem. Thanks for the explanations! Cheers, Jasmin From joe at gilith.com Wed Mar 28 02:28:06 2012 From: joe at gilith.com (Joe Hurd) Date: Tue, 27 Mar 2012 19:28:06 -0700 Subject: [metis-users] KBO vs. LPO In-Reply-To: <2BB9CBEF-48A9-4E2C-BA07-48F602671ECB@gmail.com> References: <2BB9CBEF-48A9-4E2C-BA07-48F602671ECB@gmail.com> Message-ID: Hi Jasmine, Sorry for the delay in replying. Around ten(!) years ago I carried out some experiments with term orderings in Metis, specifically the "stickiness" of constraints on clauses resulting from KBO comparisons. I wrote them up in a Cambridge CL technical report: http://www.gilith.com/research/papers/ordering.pdf However, that reference was just for interest's sake. To answer your specific question, I would certainly consider adding different orderings to Metis. In Clause.sig the order is explicitly parameterized: type parameters = {ordering : KnuthBendixOrder.kbo, orderLiterals : literalOrder, orderTerms : bool} If someone were to create a new module LexicographicPathOrder with the same signature as KnuthBendixOrder I would happily hook pull it in to the Metis sources and offer it as an option to Clause.parameters. Cheers, Joe On Mon, Mar 19, 2012 at 5:16 PM, Jasmin Blanchette wrote: > Hi Joe, > > KBO is hard-coded in Metis. Although KBO is generally acknowledged to "order more things" than LPO, there are some advantages to LPO, such as the possibility of orienting > > ? ?map f (Cons x xs) = Cons (f x) (map f xs) > > (and in general equations where variables occur several times on the right-hand side) left-to-right. Combined with an appropriate precedence function (that reflects the order in which symbols are defined in HOL), this could do wonders. > > From what I understand, it would be fairly straightforward to adapt Metis's code to support LPO as well -- or am I mistaken? What are your views on this? Would you be ready to consider an eventual patch that adds LPO, or at least the possibility of plugging-in non-KBO term orderings, for integration in the Metis repository? > > Thanks in advance. > > Cheers, > > Jasmin > > > _______________________________________________ > metis-users mailing list > metis-users at gilith.com > http://www.gilith.com/mailman/listinfo/metis-users From jasmin.blanchette at gmail.com Wed Mar 28 10:21:36 2012 From: jasmin.blanchette at gmail.com (Jasmin Blanchette) Date: Wed, 28 Mar 2012 13:21:36 +0300 Subject: [metis-users] KBO vs. LPO In-Reply-To: References: <2BB9CBEF-48A9-4E2C-BA07-48F602671ECB@gmail.com> Message-ID: Hi Joe, > Around ten(!) years ago I carried out some experiments with term > orderings in Metis, specifically the "stickiness" of constraints on > clauses resulting from KBO comparisons. I wrote them up in a Cambridge > CL technical report: > > http://www.gilith.com/research/papers/ordering.pdf > > However, that reference was just for interest's sake. To answer your > specific question, I would certainly consider adding different > orderings to Metis. In Clause.sig the order is explicitly > parameterized: > > type parameters = > {ordering : KnuthBendixOrder.kbo, > orderLiterals : literalOrder, > orderTerms : bool} > > If someone were to create a new module LexicographicPathOrder with the > same signature as KnuthBendixOrder I would happily hook pull it in to > the Metis sources and offer it as an option to Clause.parameters. Since last week, I carried out some experiments with E and SPASS, both of which support both KBO and LPO, and I wasn't able to get the expected improvements on a large set of benchmarks -- although it helped a lot for a few examples I constructed. So I'm unlikely to pursue this further. But thanks anyway! Cheers, Jasmin