Allow Bors to Auto-Retry

Development
#1

I've been a happy user of bors for the past few weeks, but and I'd like to request a new feature.

When bors fails due to a test flake, it would be nice to preconfigure bors to auto-retry the PR for some predefined number of times.

I don't think this is possible today. Would it be possible with bors?

#2

I think this could be difficult for Bors. For example, if a batch with two or more pull requests fail, how does Bors know whether it should auto-retry the batch or proceed to bisect the batch?

An easier approach will be for you to implement the auto-retry logic in your test script. Here’s an example (taken from here) of a bounded auto-retry loop. You would replace npm test with the test command(s) relevant for your project.

#!/bin/bash
sleep 1
try_count=0
while [ $try_count != 100 ]; do
	npm test
	if [ $? = 0 ]; then exit 0; fi
	try_count=`expr $try_count + 1`
	sleep 1
done
exit 1

See also @notriddle’s comment here:

One thing that has helped me with nasty CI setups (WebDriver, in my case) is to wrap the test suite in a bounded retry-loop. If your tests are flaky, that's probably a good idea no matter what you do with bors.