Pārlūkot izejas kodu

gitlab-ci: Make the test job fail when bugs are unexpectedly fixed.

If people fix bugs without updating the expected-fails list, then we
end up with a lack of coverage of those failures in the future.  Also,
some day down the line another developer ends up trying to figure out
if the bug was actually fixed or their environment is just failing to
reproduce it.

Suggested-by: Kristian H. Kristensen <hoegsberg@google.com>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Acked-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
tags/19.3-branchpoint
Eric Anholt pirms 6 gadiem
vecāks
revīzija
7859eb1390
1 mainītis faili ar 11 papildinājumiem un 0 dzēšanām
  1. 11
    0
      .gitlab-ci/deqp-runner.sh

+ 11
- 0
.gitlab-ci/deqp-runner.sh Parādīt failu

@@ -123,3 +123,14 @@ if [ -s /tmp/new-fails.txt ]; then
else
echo "No new failures"
fi

sort /tmp/case-list.txt > /tmp/sorted-case-list.txt
comm -12 /tmp/sorted-case-list.txt /tmp/expected-fails.txt > /tmp/expected-fails-in-caselist.txt
comm -13 $RESULTS/fails.txt /tmp/expected-fails-in-caselist.txt > /tmp/new-passes.txt
if [ -s /tmp/new-passes.txt ]; then
echo "Unexpected passes, please update $DEQP_EXPECTED_FAILS (or add flaky tests to $DEQP_SKIPS):"
cat /tmp/new-passes.txt
exit 1
else
echo "No new passes"
fi

Notiek ielāde…
Atcelt
Saglabāt