Create More Robust Apex Tests and More Flexible Test Runs

We’ve made lots of improvements to Apex tests and the ways you can run them.

Create Test Suites of Commonly Used Apex Test Classes

You probably have sets of Apex test classes that you regularly run together. For example, perhaps you have test classes that you run every time you prepare for a deployment or Salesforce releases a new version. You can now create test suites that contain those sets of classes. Rather than selecting all the relevant classes each time you start a test run, you can simply run one or more test suites. This feature is available in both Lightning Experience and Salesforce Classic.

Create and Run Test Suites in the Developer Console

To create a test suite using the Developer Console, select Test | New Suite. To choose which classes are in a test suite, select Test | Suite Manager | Your Test Suite | Edit Suite. You can create or modify test suites that contain up to 200 test classes using the Developer Console.

To run one or more test suites’ member classes from the Developer Console, select Test | New Suite Run.

Create and Run Test Suites in the API

Each test suite is represented in the SOAP, REST, and Tooling APIs by an ApexTestSuite object, which has a TestSuiteName and an ID. A set of TestSuiteMembership objects, each of which contains an ApexTestSuiteId and an ApexClassId, determines membership in the test suite. Create one TestSuiteMembership object for each class that you want to add to each test suite. To remove a test class from a test suite, delete its TestSuiteMembership object.

The following SOQL query returns the membership object that relates this Apex class to this test suite.
SELECT Id FROM TestSuiteMembership WHERE ApexClassId = '01pD0000000Fhy9IAC'
    AND ApexTestSuiteId = '05FD00000004CDBMA2'

To run test suites using the Tooling REST API, POST a suiteids list to the runTestsAsynchronous REST resource. You can POST both a suiteids list and a classids list to runTestsAsynchronous. However, if you send a tests array, you can’t send suiteids or classids.

To run test suites using the Tooling SOAP API, pass suite IDs to the runTestsAsynchronous() SOAP call. Either a classids or a suiteids parameter is mandatory for runTestsAsynchronous, but only one of the two is required. To provide only one, specify the other as null.

Stop a Test Run That’s Failing Miserably

Large runs of Apex tests sometimes take longer than we’d like them to. Waiting for a run to finish only to learn that many of your tests failed can be irritating. You can now configure test runs to stop executing new tests after a given number of tests fail. You no longer need to waste your time waiting for the results of a run that you’re going to rerun after fixing issues in your org. You can set how many tests can fail in test runs that you execute both in the Developer Console and using the Tooling API. This feature is available in both Lightning Experience and Salesforce Classic.

Choose a Test Failure Threshold

To allow all tests in your org to run, either don’t set a test failure threshold or set the threshold to -1. To stop your test run from executing new tests after a given number of tests fail, set the failure threshold to an integer value from 0 to 1,000,000. This integer value sets the maximum allowable test failures. A value of 0 causes the test run to stop if any failure occurs. A value of 1 causes the test run to stop on the second failure, and so on.

Synchronous test runs stop when the failure threshold is exceeded. However, asynchronous test runs execute tests in parallel, so they take a bit longer to stop. An asynchronous test run stops after it finishes executing the tests that are in progress when the failure threshold is exceeded.

Keep in mind that high values can cause slow performance. Each 1,000 tests that you allow to fail adds about 3 seconds to your test run, not including the time that the tests take to execute.

Set Allowed Test Failures in the Developer Console

The Developer Console’s test run configuration panes have a new Settings button. To specify how many tests can fail before a test run or test suite run is canceled, click Settings, enter a value for Number of failures allowed, and then click OK. This value applies for all test runs that you execute, until you close the Developer Console or set a new value.

Set maxFailedTests Using the Tooling API

You can set a maxFailedTests value for a Tooling API REST resource or SOAP call. To use REST, POST the optional maxFailedTests parameter to the runTestsAsynchronous resource. To use SOAP to limit failed tests in a synchronous test run, send a value for RunTestsRequest.maxFailedTests to the runTests() call. To use SOAP to limit failed tests in an asynchronous test run, send a maxFailedTests value above -1 to the runTestsAsynchronous() call.

Set and Modify the CreatedDate Field in Apex Tests

In response to your IdeaExchange request, we’ve added the System.Test.setCreatedDate method. This method sets a Datetime value for a test-context sObject’s CreatedDate field.
Insert your test record before you set its CreatedDate, as shown in this example.
@isTest 
private class SetCreatedDateTest {
    static testMethod void testSetCreatedDate() {
        Account a = new Account(name='myAccount');
        insert a;
        Test.setCreatedDate(a.Id, DateTime.newInstance(2012,12,12));
        Test.startTest();
        Account myAccount = [SELECT Id, Name, CreatedDate FROM Account 
                             WHERE Name ='myAccount' limit 1];
        System.assertEquals(myAccount.CreatedDate, DateTime.newInstance(2012,12,12));
        Test.stopTest();
    }
}

Call Test.startTest() to Reliably Reset Limits in Apex Tests

A block of test code enclosed by the Test.startTest() and Test.stopTest() methods now reliably receives its own block of governor limits. Test.startTest() stores your per-transaction limit counters and temporarily resets them to zero. Test.stopTest() restores your limit counters to pre-Test.startTest() values. When your test method finishes, all per-transaction limits reset to zero. Previously, some limits, such as for SOQL queries, didn’t always reset inside a Test.startTest()/Test.stopTest() block.

Use @future to Avoid the Dreaded MIXED_DML_OPERATION Error in Apex Tests

Mixed Data Manipulation Language (DML) operations within a single transaction aren’t allowed. You can’t perform DML on a setup sObject and a non-setup sObject in the same transaction. Instead, you can perform one type of DML as part of an asynchronous job and run the other types in other asynchronous jobs or in the original transaction. This alternative is widely used. However, when this alternative was used within the context of an Apex test, errors still resulted. We’ve fixed this bug.

This example previously didn’t work in a test context, but now it does. UserAndContactTest.testUserAndContact() calls the future method InsertFutureUser.insertUser() to insert a user. UserAndContactTest.testUserAndContact() then inserts a contact.

@isTest
public class UserAndContactTest {
    public testmethod static void testUserAndContact() {
        InsertFutureUser.insertUser();
        Contact currentContact = new Contact(
            firstName = String.valueOf(System.currentTimeMillis()),
            lastName = 'Contact');
        insert(currentContact);
    }
}
public class InsertFutureUser {
    @future
    public static void insertUser() {
        Profile p = [SELECT Id FROM Profile WHERE Name='Standard User'];
        UserRole r = [SELECT Id FROM UserRole WHERE Name='COO'];
        User futureUser = new User(firstname = 'Future', lastname = 'User',
            alias = 'future', defaultgroupnotificationfrequency = 'N',
            digestfrequency = 'N', email = 'test@test.org',
            emailencodingkey = 'UTF-8', languagelocalekey='en_US', 
            localesidkey='en_US', profileid = p.Id, 
            timezonesidkey = 'America/Los_Angeles',
            username = 'futureuser@test.org',
            userpermissionsmarketinguser = false,
            userpermissionsofflineuser = false, userroleid = r.Id);
        insert(futureUser);
    }
}

Compare Currency to Decimals in Apex Tests

We’ve fixed a bug that caused an exception to be thrown when comparing a Currency value to a Decimal value in an Apex test. You can now use data silos that contain both Currency values and Decimal values and compare these data types to your heart’s content.

Test WSDL-Based Asynchronous Callouts

Previously, you could test synchronous WSDL-based callouts, or you could test asynchronous non-WSDL-based callouts. But testing asynchronous callouts made from an imported WSDL caused your tests to fail with an internal Salesforce error. We’ve fixed this bug.