I've been a happy user of bors for the past few weeks, but and I'd like to request a new feature.
When bors fails due to a test flake, it would be nice to preconfigure bors to auto-retry the PR for some predefined number of times.
I don't think this is possible today. Would it be possible with bors?
I think this could be difficult for Bors. For example, if a batch with two or more pull requests fail, how does Bors know whether it should auto-retry the batch or proceed to bisect the batch?
An easier approach will be for you to implement the auto-retry logic in your test script. Here’s an example (taken from here) of a bounded auto-retry loop. You would replace
npm test with the test command(s) relevant for your project.
while [ $try_count != 100 ]; do
if [ $? = 0 ]; then exit 0; fi
try_count=`expr $try_count + 1`
See also @notriddle’s comment here:
One thing that has helped me with nasty CI setups (WebDriver, in my case) is to wrap the test suite in a bounded retry-loop. If your tests are flaky, that's probably a good idea no matter what you do with bors.