Ignore particular tests on failure with JUnit

    Why on earth would you like to ignore some tests if they failed?! - you may ask. If they failed they must be red and they must draw your attention or even cut your fingers. In general, it is true. However, there is no ALWAYS and there is no NEVER.

You may have some tests which call external resources. Of course they are not unit tests but rather integration or functional ones. What if external resource is not available for some reason (e.g. network connection problem or ongoing redeployment of external system)? Do you want to wait another hour or so to possibly obtain a green build and final artifacts? Do you want to suffer because of a fault that is out of your reach? If you agree please abandon further reading of this post :).

The idea is to provide a mechanism which will ignore tests (the ones calling external resources) if they throw particular exception. There are flags exposed by Maven:
-Dmaven.test.failure.ignore=true or -DtestFailureIgnore=true
but they ignore all tests failures.
My implementation is based on JUnit Assumption mechanism and JUnit Rule mechanism. If AssumptionViolatedException is thrown from a test - test is not consider as failed but as ignored. The implementation is as follows:

public class IgnoreOnFailureRule implements TestRule
{
    @Override
    public Statement apply(Statement base, Description description)
    {
        return new IgnoreOnFailureStatement(base, description);
    }

    private class IgnoreOnFailureStatement extends Statement
    {
        private Statement base;
        private Description description;

        private IgnoreOnFailureStatement(Statement base, Description description)
        {
            this.base = base;
            this.description = description;
        }

        @Override
        public void evaluate() throws Throwable
        {
            IgnoreIf ignoreIf = checkIgnoreIfExistence();
            try
            {
                base.evaluate();
            }
            catch (Exception ex)
            {
                if (exceptionShouldBeIgnored(ex, ignoreIf.anyException()))
                {
                    throw new AssumptionViolatedException("Exception while executing test. Test is ignored!", ex);
                }
                throw ex;
            }
        }

        private IgnoreIf checkIgnoreIfExistence()
        {
            IgnoreIf ignoreIf = description.getAnnotation(IgnoreIf.class);
            if (ignoreIf == null)
            {
                throw new IllegalStateException(IgnoreOnFailureRule.class.getSimpleName() + 

                " can only be used in pair with @" + IgnoreIf.class.getSimpleName());
            }
            return ignoreIf;
        }

        ...
    }
}

@Retention(RetentionPolicy.RUNTIME)
@Target({ ElementType.METHOD })
public @interface IgnoreIf 
{
    Class[] anyException();
}

and the usage example:

public class ExternalResourceIT
{
    @Rule
    public IgnoreOnFailureRule ignoreOnFailureRule = new IgnoreOnFailureRule();
    
    @Test
    @IgnoreIf(anyException = ExternalResourceException.class)
    public void shouldTestExternalResource()
    {
        // given
        ...
        // when
        ...
        //then
        ...
    }
}

If the exception is thrown - test is ignored. It is not red but yellow (thin yellow line at the bottom):

Thanks to that you still know that something went wrong but the build is successful. If the test passes - it will be green. If the test throws an exception different than the one defined within @IgnoreIf annotation - it will be red.
The final note is to use this mechanism with caution, otherwise you may end up with yellow builds rather than green ;).