Creating An Adapter

We’re going to walk through how to tackle building a Valence Adapter. What to think about, where to start, what’s important.

If you haven’t already read the primer on how Valence works overall, definitely read that first. Also make sure you have read Extensions and Configurability.

Ok, up to speed? Let’s get to work.

The Beginning

An Adapter’s job is to be the liaison between Valence and External Systems that can share or accept data. Valence and the Adapter work together as partners to strategize about the best way to tackle a goal, and then collaborate to execute on it. If you want to take a peek at some existing Adapters as you read this article, you can find them on GitHub.

Any Valence extension Apex class is going to implement various Valence-provided interfaces in order to be a cog in the Valence machine. These interfaces are like a contract: they define what Valence will provide to you, and also what Valence expects from you. Each interface you implement is a signal to Valence that you support another facet of the Adapter-Valence partnership; just implementing the interface itself is a way of registering your Adapter for that behavior.

It’s always good to start small and then build up from there. The first prototype of your Adapter probably implements authentication, schema, and one source or target interface.

Your Adapter never has to deal with orchestration, or timing, or job state. You don’t have to think about what execution context you’re in, how to recover the job if it fails, if there’s already been DML before your callout or not, or if this is a Queueable or a Batch or synchronous. Valence takes care of all of that. Each time Valence interacts with your Adapter you are given exactly what you need to know, and asked for simple, finite things (like the next batch of records).

An Adapter is a distillation of just the unique business logic that is specific to interacting with each external system. You are sitting on top of 95% of the work of creating an integration already being done, and you are writing the last mile.

Authentication

Every Adapter has to figure out how it is going to handle authenticating with the systems it talks to, so we’ve got an extensive writeup on what that looks like.

Adapter Interface: NamedCredentialAdapter

Schema

Almost always, Adapters expose a Schema so that admin users can select which table they want to interact with, and also can see what fields exist on each table.

Adapter Interface: SchemaAdapter

Valence asks you for schema details but how you go about answering that question is encapsulated in your Adapter and up to you. Here are some viable approaches, increasing in sophistication:

  • The external system your Adapter interacts with has a very simple schema, so you’ve just hardcoded your getTables() and getFields() methods with the schema and called it a day.

  • The external system has a relatively static schema but it’s rather large, so you ship a flat file (CSV, JSON, etc) with your Adapter as a static resource and then inspect it at runtime when you are asked for schema info.

  • The external system has some kind of reflective or discovery endpoint where you can ask it about its tables and fields, so when Valence asks you for schema details you make an HTTP callout to fetch them.

We always prefer that your schema details be dynamic (option #3) so that when an external system’s structure is altered, no coding or file changes are needed before admin users can see and work with those changes. Depending on the API you are working with this may not be possible.

Note

Technically it is possible for an Adapter to have no schema. An Adapter that does not implement SchemaAdapter is assumed to basically have one table that is always the one that is read from or written to. The Adapter can still be used as a source or a target (instead of the admin user picking a table they just pick the adapter and move to the next step). Fields are still needed, of course, to do mappings, but instead of the Adapter reporting the fields Valence will detect them during record flow, or they can be specified using ValenceField__mdt instances.

Direction

The next things to figure out is in what direction can your Adapter move data. Is it an Adapter that reads data from an external system (a “source Adapter”)? Does it write to an external system (a “target Adapter”)? Does it do both? Whenever an admin user sets up a Link they select an Adapter as their source Adapter from the list of source Adapters registered in the org, and a target Adapter from the list of target Adapters registered in the org.

It’s perfectly fine to have one Apex class paired with one cMDT registration record that handles multiple source Adapter behaviors and is also a target Adapter. It’s also perfectly fine to split these up into multiple classes if that’s what makes sense for your circumstances.

Being a Data Source

If you are a source Adapter, your basic responsibility is to produce RecordInFlight instances that represent records from an external system. That’s pretty much it! Everything else is just about handling the nuance of accomplishing that.

A RecordInFlight is really just a fancy Map of key-value pairs. You build them from a Map<String, Object>, like this:

valence.RecordInFlight sample = new valence.RecordInFlight(new Map<String, Object> {'first_name' => 'Tom', 'last_name' => 'Smith', 'opt_in' => true});

Most of the work of being a data source is understanding which records are needed, and obtaining them in some way.

Operation

Each RecordInFlight has an operation, which is a String value that indicates what should be done with this record. We recommend supporting these two operations at a minimum, but you can use more as long as both source and target Adapter know about them:

  • “upsert”

  • “delete”

The operation value is set by the source Adapter (which knows why this record is being transmitted) and consumed by the target Adapter so that it can apply the appropriate action to the record in the target external system.

  • If you don’t specify an operation on your records, the default is “upsert”.

  • You can mix operations in the same list of records.

Basic Source Adapter Interfaces

These interfaces are your basic building blocks for source Adapters. Each allows your Apex class to be used in a different style of Link run.

Tip

Any Adapter that implements SourceAdapterForPull should also immediately implement SourceAdapterScopeSerializer. It’s easy to implement and it allows Links that use your source Adapter to run in parallel mode, which is much, much faster.

Source Adapter Enhancement Interfaces

These interfaces build on top of the basic ones, and register your Adapter either for extra behavior it needs from Valence, or for extra functionality it can offer Valence.

  • ChainFetchAdapter - Implement if your Adapter is fetching data from a system that cannot predict in advance how many records (or how many batches) will be needed to retrieve all the records. This interface allows you to alternate record fetches with record processing indefinitely until all source records are exhausted.

  • ConfigurableSourceAdapter - Implement if your source Adapter is user-configurable (see Configurability).

  • DelayedPlanningAdapter - Implement if your Adapter needs a bit of real-world time between when it is asked for data and when it is ready to serve that data.

Being a Data Target

If you are a target Adapter, your basic responsibility is to write RecordInFlight instances to an external system, and mark them up with the results of that operation.

A RecordInFlight is really just a fancy Map of key-value pairs. You can extract the Map<String, Object> that was constructed for you to deliver like this:

Map<String, Object> oneRecord = aRecordInFlight.getProperties();

The collection of RecordInFlight instances you are handed is a live collection, and any changes you make to them will be conveyed to Valence. This means as you process the collection and try to write it to your adapter system, you should link those results back to each individual RecordInFlight so you can tell Valence how it went.

Example of linking API response to each record
List<Map<String,Object>> rawData = new List<Map<String, Object>>();
for(valence.RecordInFlight record : records) {
        rawData.add(record.getProperties());
}

List<Result> results = sendToAPI(rawData);

for(Integer i = 0, j = results.size(); i < j; i++) {

        records[i].setCreated(results[i].createdNewRecord);

        records[i].setSuccess(results[i].success);

        if(results[i].success == false) {
                records[i].addError(results[i].errorMessage);
        }
}

private class Result {
        private Boolean success;
        private Boolean createdNewRecord;
        private String errorMessage;
}

Operation

Adding on to our discussion of Record Operation from earlier, it is the target Adapter’s responsibility to apply the correct action that a record needs.

Don’t assume all records you receive are upserts, some might be deletes. Typically you’ll end up sorting RecordInFlight instances into different buckets based on their operation and resolve them separately.

If you don’t support deleting records, mark those records in some way (probably with something like record.ignore('Delete operation not supported.')).

Not all admins want to allow records to be deleted, so you may want to add a configuration option to your Adapter to let an admin toggle deletes on or off.

Basic Target Adapter Interfaces

Target Adapters are a bit simpler than source Adapters in that there is only one basic interface:

  • TargetAdapter - Implement if your Adapter can be the endpoint of a Link, i.e. the place where records are sent at the end of processing.

Warning

It is a requirement that your target Adapter respects the Link setting for Testing Mode, which an admin user sets when they don’t want records to actually persist into the target system.

You will know if a Link run is in testing mode if the LinkContext testingMode boolean property is set to true. At a minimum, you can simply immediately return from pushRecords():

public void pushRecords(valence.LinkContext context, List<valence.RecordInFlight> records) {

        // don't send any data if this Link is running in testing mode
        if(context.testingMode == true) {
                return;
        }

If possible, it’s nice if you have a mechanism to do a trial push where you can test a write but roll back so that you can attach any errors or warnings to the RecordInFlight instances. Very few APIs support a mechanic like this, so we don’t expect it but it’s a nice to have.

Target Adapter Enhancement Interfaces

Handling Errors

Every Adapter may potentially run into an exception or error when working with records.

If the issue is isolated to a single record and you can proceed, flag that record:

try {
        doSomethingWithARecord(record);
} catch(Exception e) {
        record.addError('Failed to flim flam the jibber jabber', e);
}

If the issue is catastrophic and your Adapter cannot continue, throw a valence.AdapterException:

throw new valence.AdapterException('Things have gone very badly wrong.');

If you have a causing Exception, you can wrap it:

throw new valence.AdapterException('Things have gone very badly wrong.', exceptionThatCausedIt);

Test Coverage

When you’re ready to start writing test coverage for your Adapter, check out Writing Test Coverage.

Summary

Adapters are the lifeblood of Valence, and as you can see there’s a huge amount of tooling and support to help them shine. There’s very few edge cases or situations where there is not already a feature or interface that will help you accomplish your goals.

Don’t forget to take a peek at our open-source Adapters to see if what you need already exists, or to get inspiration and guidance on how to build them.