Building a Configurable Filter that Ignores Records Based on User-selected Cutoff Date

Here we’ll take a slightly more advanced scenario and walk through it.

Desired Behavior

A User can apply a condition to an incoming date field where that field is evaluated and if it fails our test, the record is ignored.
The test will be comparing the record’s date in the field to a cutoff/threshold date that has been configured by the User.
A User should be able to apply this condition to any incoming date field, and (if desired) more than one date field on the same record.
The cutoff dates are configured per-mapping, so if multiple fields should be inspected they can have different cutoff dates.

Solution Walkthrough

To satisfy these expectations we’ll want to create a custom Filter that implements both TransformationFilter and ConfigurablePerMappingFilter.

Our configuration needs are pretty simple (just a date picker), so we’ll use the Configuration Structure pattern for handling configurations.

Class Declaration

We start out with our class declaration and implementing both of our interfaces.

/**
 * Valence filter that allows us to set a date threshold that will cause records to be ignored if a given
 * field from that record is older than our threshold.
 */
global with sharing class IgnoreOldRecordsFilter implements valence.TransformationFilter, valence.ConfigurablePerMappingFilter {

Warning

Make sure you declare your class as global, otherwise Valence won’t be able to see it and use it!

Configuration Setup

Since we opted for a configuration structure, we’ll be returning null from getMappingConfigurationLightningComponent().

The shape we return from getMappingConfigurationStructure() will be used by Valence to build a UI on our behalf and show it to the User. We are going to use the “date” flavor of lightning:input by setting the “type” attribute on that base component so that our User gets a nice, friendly date picker.

Valence will save the User-selected date to the database for us, and we’ll be given the value back later when we need it.

    public String getMappingConfigurationLightningComponent() {
        return null;
    }

    public String getMappingConfigurationStructure() {
		return DynamicUIConfigurationBuilder.create('Select a date below. Any records that have a value in this mapping older than the selected date will be ignored (exact date matches are not ignored).')
			.addField(
				DynamicUIConfigurationBuilder.createField('cutoff')
				.addAttribute('label', 'Cutoff Date')
				.addAttribute('type', 'date')
			)
			.finalize();
    }

Tip

We don’t have to set “componentType” on the cutoff field because lightning:input is the default component type.

Configuration Explanation

We always want to give the User useful information about what they’ve done and what they can expect. Valence uses explainMappingConfiguration() to give us an opportunity to interpret a configuration and break it down in plain language for the User.

    public String explainMappingConfiguration(String configuration) {

        String explanation = 'This Filter will set records to be ignored if their field value (the one in this mapping) is older than {0}.';

        Configuration config = (Configuration)JSON.deserialize(configuration, IgnoreOldRecordsFilter.Configuration.class);

        return String.format(explanation, new List<String>{String.valueOf(config.cutoff)});
    }

Configuration Class

For convenience and cleanliness it’s a good idea to create a simple inner Apex class to hold your configuration structure. Valence serializes configuration values from the form the User filled out into a JSON object whose keys are the name values you specified in your configuration schema. In our case we defined a single field called cutoff that we expect to find a serialized Date value inside.

    /**
     * Simple class for holding the configuration needed for this filter.
     */
    private class Configuration {
        private Date cutoff;

Restricting Filter Usage

Some Filters only make sense in specific scenarios, for example RelationshipFilter (the built-in Valence Filter that handles populating Master-Detail and Lookup fields) only makes sense for records flowing into Salesforce, not outbound.

For this cutoff Filter we are building, we aren’t going to restrict it to only certain Links. All Links can use it.

    public Boolean validFor(valence.LinkContext context) {
        return true;
    }

Processing Records

Finally, we get into the core purpose of our Filter: ignoring old records. Let’s walk through our process() method.

Set up a Map we will use to line up the names of the record fields we’re going to inspect with the configured cutoff date for each field.

    public void process(valence.LinkContext context, List<valence.RecordInFlight> records) {

        Map<String, Date> cutoffsBySourceField = new Map<String, Date>();

Iterate through the Mapping instances we are given as part of the LinkContext. Remember that Valence is clever here and will inject serialized User configurations from the database into mapping.configuration properties wherever the User has set up a configuration.
We collect the cutoff Date values from a deserialized Configuration instance for any populated configurations.

        for(valence.Mapping mapping : context.mappings.values()) {

            // skip blank configurations
            if(String.isNotBlank(mapping.configuration)) {
                Configuration config = (Configuration)JSON.deserialize(mapping.configuration, IgnoreOldRecordsFilter.Configuration.class);
                cutoffsBySourceField.put(mapping.sourceFieldName, config.cutoff);
            }
        }

Now that we’ve assembled our Map, if it’s empty we can stop processing.

        // bail out if we didn't find any
		if(cutoffsBySourceField.isEmpty()) {
			return;

Now we iterate through the incoming RecordInFlight instances.

        for(valence.RecordInFlight record : records) {

For each field we need to check, inspect that field’s value for this record.

            for(String sourceField : cutoffsBySourceField.keySet()) {
                Date cutoff = cutoffsBySourceField.get(sourceField);

Compare the field value to our cutoff date for this field. If older than the cutoff, mark this record as ignored.

                Long fieldValue = (Long)record.getOriginalProperties().get(sourceField);
                if(fieldValue != null) {
                    if(Datetime.newInstance(fieldValue).dateGmt() < cutoff) {
                        record.ignore('Record field <' + sourceField + '> older than cutoff of ' + cutoff);

Hint

You can see in the code for this example scenario we are assuming that all dates are being transmitted as Long values, i.e. milliseconds since Epoch. This is a simplification and you may not be able to make this same assumption in your real-world scenario!

Full Solution Code

Here is the complete solution code that we walked through above.

/**
 * Valence filter that allows us to set a date threshold that will cause records to be ignored if a given
 * field from that record is older than our threshold.
 */
global with sharing class IgnoreOldRecordsFilter implements valence.TransformationFilter, valence.ConfigurablePerMappingFilter {

    public String getMappingConfigurationLightningComponent() {
        return null;
    }

    public String getMappingConfigurationStructure() {
		return DynamicUIConfigurationBuilder.create('Select a date below. Any records that have a value in this mapping older than the selected date will be ignored (exact date matches are not ignored).')
			.addField(
				DynamicUIConfigurationBuilder.createField('cutoff')
				.addAttribute('label', 'Cutoff Date')
				.addAttribute('type', 'date')
			)
			.finalize();
    }

    public String explainMappingConfiguration(String configuration) {

        String explanation = 'This Filter will set records to be ignored if their field value (the one in this mapping) is older than {0}.';

        Configuration config = (Configuration)JSON.deserialize(configuration, IgnoreOldRecordsFilter.Configuration.class);

        return String.format(explanation, new List<String>{String.valueOf(config.cutoff)});
    }

    public Boolean validFor(valence.LinkContext context) {
        return true;
    }

    public void process(valence.LinkContext context, List<valence.RecordInFlight> records) {

        Map<String, Date> cutoffsBySourceField = new Map<String, Date>();

        /*
         * Assemble any cutoffs that have been configured by admins.
         */
        for(valence.Mapping mapping : context.mappings.values()) {

            // skip blank configurations
            if(String.isNotBlank(mapping.configuration)) {
                Configuration config = (Configuration)JSON.deserialize(mapping.configuration, IgnoreOldRecordsFilter.Configuration.class);
                cutoffsBySourceField.put(mapping.sourceFieldName, config.cutoff);
            }
        }

        // bail out if we didn't find any
		if(cutoffsBySourceField.isEmpty()) {
			return;
		}

        /*
         * Iterate through our records, ignoring where appropriate based on cutoff dates.
         */
        for(valence.RecordInFlight record : records) {
            for(String sourceField : cutoffsBySourceField.keySet()) {
                Date cutoff = cutoffsBySourceField.get(sourceField);
                Long fieldValue = (Long)record.getOriginalProperties().get(sourceField);
                if(fieldValue != null) {
                    if(Datetime.newInstance(fieldValue).dateGmt() < cutoff) {
                        record.ignore('Record field <' + sourceField + '> older than cutoff of ' + cutoff);
                    }
                }
            }
        }
    }

    /**
     * Simple class for holding the configuration needed for this filter.
     */
    private class Configuration {
        private Date cutoff;
    }
}