COMP 105 fault graph for type inferrers

Test results in matrix form are on another page.

The graph below shows the results of testing type inferrers.
Read the nodes as follows:

The number or numbers in each node identify a submission or set of submissions. Numbers are chosen to keep students anonymous; you should have received your number by email.
Each node shows how many tests were passed by submissions in that node. The graph includes only meaningful tests. A test is meaningful if it distinguishes a unique set of outcomes, and if its results are not implied by a combination of other tests. Most outcomes have a set of equivalent tests which produce identical outcomes; we use the smallest such test.
The bars at the bottom of each node show exactly which tests were passed. A vertical bar represents a pass; a dot represents failure or a catastrophic error.

The edges show which solutions are strictly better than which other solutions; better solutions appear higher in the graph.

If node A points to node B, than node B passed strictly more tests than node A; node B is strictly better. The edge from A to B shows the smallest test case that A fails and B passes.
If there is a single node at the top, which all the others point to, it contains the submissions that passed all tests.
If two nodes are not connected by any path, then the submissions in those nodes have distinct, independent bugs.
If there is a node not at the top which contains more than one submission, it means that as far as we could discover by testing, these submissions have identical faults.

If the graph is blurry or pixelated, click on it to get Scalable Vector Graphics, then zoom. From left to right, the tests in the graph exposed these faults:

1a. Term (begin) should have type unit, but your interpreter failed with an uncaught exception.
2a. Term (car 3) should trigger a uML error, but your interpreter failed with an uncaught exception.
3a. Term (cons (+ 1 2) '()) should have type int list, but your interpreter failed with an uncaught exception.
3b. Term (cons (+ 1 2) '()) should have type int list, but your interpreter failed with the message type error: Cannot unify int * int and int * int.
4a. Term (val lists '(() ())) should have type forall 'a . 'a list list, but your interpreter failed with an uncaught exception.
4b. Term (val lists '(() ())) should have type forall 'a . 'a list list, but your interpreter failed with a uML error message.
5a. Term (val lists2 '((a) (1))) should trigger a uML error, but your interpreter failed with an uncaught exception.
5b. Term (val lists2 '((a) (1))) should trigger a uML error, but your interpreter inferred type sym list list.
6a. Term (val lists3 '(() (1) ())) should have type int list list, but your interpreter failed with an uncaught exception.
6b. Term (val lists3 '(() (1) ())) should have type int list list, but your interpreter inferred type forall 'a . 'a list list.
6c. Term (val lists3 '(() (1) ())) should have type int list list, but your interpreter failed with a uML error message.
7a. Term (lambda (l) (if (car l) '() '())) should have type forall 'a . bool list -> 'a list, but your interpreter failed with an uncaught exception.
7b. Term (lambda (l) (if (car l) '() '())) should have type forall 'a . bool list -> 'a list, but your interpreter inferred type forall 'a, 'b . 'a -> 'b list.
8a. Term (define f (x) (if x (car x) (f x))) should trigger a uML error, but your interpreter failed with an uncaught exception.
8b. Term (define f (x) (if x (car x) (f x))) should trigger a uML error, but your interpreter inferred a type for it.
9a. Term (lambda (x y z) (cons (cons x y) z)) should have type forall 'a . 'a * 'a list * 'a list list -> 'a list list, but your interpreter failed with an uncaught exception.
9b. Term (lambda (x y z) (cons (cons x y) z)) should have type forall 'a . 'a * 'a list * 'a list list -> 'a list list, but your interpreter inferred type forall 'a . 'a list list * 'a list * 'a -> 'a list list.
10a. Term (lambda (x) (begin (+ x 1) (if x 0 1))) should trigger a uML error, but your interpreter failed with an uncaught exception.
10b. Term (lambda (x) (begin (+ x 1) (if x 0 1))) should trigger a uML error, but your interpreter inferred type forall 'a . 'a -> int.
11a. Term (lambda (x) (let ((b (+ x 1))) (and (= b 1) x))) should trigger a uML error, but your interpreter failed with an uncaught exception.
11b. Term (lambda (x) (let ((b (+ x 1))) (and (= b 1) x))) should trigger a uML error, but your interpreter inferred type int -> bool.
12a. Term (val poly (letrec ((id (lambda (x) x))) (pair (id 0) (id #t)))) should have type int * bool, but your interpreter failed with an uncaught exception.
12b. Term (val poly (letrec ((id (lambda (x) x))) (pair (id 0) (id #t)))) should have type int * bool, but your interpreter failed with the message type error: Cannot unify int and bool.
13a. Term (letrec ((x (lambda (y) y))(z (lambda (y) (x y)))) (z 3)) should have type int, but your interpreter failed with an uncaught exception.
13b. Term (letrec ((x (lambda (y) y))(z (lambda (y) (x y)))) (z 3)) should have type int, but your interpreter inferred type forall 'a . 'a.
13c. Term (letrec ((x (lambda (y) y))(z (lambda (y) (x y)))) (z 3)) should have type int, but your interpreter failed with the message variable x not found.
14a. Term (letrec ((func (lambda (a b num) (if a b (+ 1 num))))) func) should have type bool * int * int -> int, but your interpreter failed with an uncaught exception.
14b. Term (letrec ((func (lambda (a b num) (if a b (+ 1 num))))) func) should have type bool * int * int -> int, but your interpreter inferred type forall 'a . bool * int * int -> 'a.
15a. Term (letrec ((f (lambda (x) (g))) (g (lambda (x) (f)))) (o g f)) should trigger a uML error, but your interpreter failed with an uncaught exception.
15b. Term (letrec ((f (lambda (x) (g))) (g (lambda (x) (f)))) (o g f)) should trigger a uML error, but your interpreter inferred type forall 'a, 'b . 'a -> 'b.
16a. Term (define test3 (x) (if (= 0 x) 0 (+ x (test3 o test3) (- x 1)))) should trigger a uML error, but your interpreter failed with an uncaught exception.
17a. Term (let ((symnil (cdr (list1 'a))) (x 1)) (begin (cons x symnil))) should trigger a uML error, but your interpreter failed with an uncaught exception.
17b. Term (let ((symnil (cdr (list1 'a))) (x 1)) (begin (cons x symnil))) should trigger a uML error, but your interpreter inferred type int list.
18a. Term (letrec ((f (lambda (x) (+ 1 x))) (g (lambda (x) (f (not x))))) g) should trigger a uML error, but your interpreter failed with an uncaught exception.
18b. Term (letrec ((f (lambda (x) (+ 1 x))) (g (lambda (x) (f (not x))))) g) should trigger a uML error, but your interpreter inferred a type for it.
19a. Term (define f (x) (let ((g (lambda (x) (f x)))) (g f))) should trigger a uML error, but your interpreter failed with an uncaught exception.
19b. Term (define f (x) (let ((g (lambda (x) (f x)))) (g f))) should trigger a uML error, but your interpreter inferred type forall 'a, 'b . 'a -> 'b.
20a. Term (define g (x) (let ((a (+ x 1)) (b (and x #t))) x)) should trigger a uML error, but your interpreter failed with an uncaught exception.
20b. Term (define g (x) (let ((a (+ x 1)) (b (and x #t))) x)) should trigger a uML error, but your interpreter inferred type bool -> bool.
21a. Term (letrec ((+ (lambda (x y) (and x y))) (foo (lambda (x y) (+ x y)))) (foo 1 2)) should trigger a uML error, but your interpreter failed with an uncaught exception.
21b. Term (letrec ((+ (lambda (x y) (and x y))) (foo (lambda (x y) (+ x y)))) (foo 1 2)) should trigger a uML error, but your interpreter inferred a type for it.
22a. Term (lambda (x) (letrec ((f (lambda (a) (cons (g a) '()))) (g (lambda (b) (cons (f b) '())))) (f x))) should trigger a uML error, but your interpreter failed with an uncaught exception.
22b. Term (lambda (x) (letrec ((f (lambda (a) (cons (g a) '()))) (g (lambda (b) (cons (f b) '())))) (f x))) should trigger a uML error, but your interpreter inferred type forall 'a, 'b . 'a -> 'b list list list.
23a. Term (define f (x) (letrec ((foo (lambda (vs) (if (null? vs) (+ x 1) (foo vs))))) (if x (foo '(1 2 3)) (foo '(4 5 6))))) should trigger a uML error, but your interpreter failed with an uncaught exception.
23b. Term (define f (x) (letrec ((foo (lambda (vs) (if (null? vs) (+ x 1) (foo vs))))) (if x (foo '(1 2 3)) (foo '(4 5 6))))) should trigger a uML error, but your interpreter inferred type forall 'a, 'b . 'a -> 'b.

Back to the COMP 105 home page.