FetchStrategy¶
FetchStrategy allows your source Adapter to tell Valence how best to ask your Adapter for records. It is an implementation of the Strategy Pattern.
This class is only used by Adapters that implement the SourceAdapterForPull interface.
There are a few different “strategies” available for your selection. You can even mix and match them depending on what’s going on, perhaps using IMMEDIATE if there are only a few records, or SCOPES if you have many thousand to retrieve.
Tip
Every strategy (except NO_RECORDS) has an alternate factory method that accepts an extra parameter value called “expectedTotalRecords”. If during planning you know exactly how many records you intend to fetch, always use this version of the method. This helps users understand how many records are going to be fetched during a Link run. If you don’t know how many records you’ll be fetching ahead of time, that’s perfectly fine, just use the simpler version of the method.
Immediate¶
The IMMEDIATE strategy calls fetchRecords() immediately in the same execution context (any state in your Adapter is still there). A null value is passed as the scope parameter.
This is the simplest and easiest strategy to work with. You can get some decent mileage out of it before you have to move to SCOPES.
One nice thing is that Valence abstracts away some of the Salesforce limits, so for example you can return more than 10,000 records from fetchRecords() (which would normally hit the DML row limit) and that’s not a problem.
You do, however, still have to stay under the heap size limit of 12 MB. If you think you might hit this limit, consider using SCOPES and breaking your result set down across multiple execution contexts.
Definition¶
global static FetchStrategy immediate();
global static FetchStrategy immediate(Long expectedTotalRecords);
Example Usage¶
public valence.FetchStrategy planFetch(valence.LinkContext context) {
return valence.FetchStrategy.immediate();
}
public List<valence.RecordInFlight> fetchRecords(valence.LinkContext context, Object scope) {
// retrieve some records
List<valence.RecordInFlight> records = goGetRecords();
return records;
}
Scopes¶
Warning
Reminder: unlike IMMEDIATE, when fetchRecords() is called on your Adapter it happens in an entirely new execution context. This means your Adapter state is totally wiped clean between calls! If you need something, put it into your scopes variable during planning.
The SCOPES strategy leverages Batch Apex to break up fetching records from the external system into multiple Salesforce execution contexts (one per scope). Each scope will have a fresh set of limits and state. In order to use this strategy, during your planFetch() call you need to figure out how many scopes are necessary. Compare context.batchSizeLimit to whatever hard limits your external system has, and use the lower of the two as your batch size.
During planning your goal is to build a list of scopes that give you whatever details you need to be able to retrieve each batch of records from the external system. Maybe that’s a unique identifier for the job combined with an offset value or page number. It varies but the scope shape is entirely up to you.
Definition¶
global static FetchStrategy scopes(List<Object> scopes);
global static FetchStrategy scopes(List<Object> scopes, Long expectedTotalRecords);
Example Usage¶
public valence.FetchStrategy planFetch(valence.LinkContext context) {
String requestId = getRequest(); // some method to get whatever info you need about the request
Integer total = countExternalRecords(); // and grab a record count, for example
// determine how many records you can fetch at a time
Integer batchSize = context.batchSizeLimit < EXTERNAL_LIMIT ? context.batchSizeLimit : EXTERNAL_LIMIT;
// build our list of custom scopes
List<MyScope> scopes = new List<MyScope>();
Integer offset = 0;
while(offset < total) {
scopes.add(new MyScope(requestId, offset));
offset += batchSize;
}
// tell Valence we're using the SCOPES strategy
return valence.FetchStrategy.scopes(scopes, total);
}
public List<valence.RecordInFlight> fetchRecords(valence.LinkContext context, Object scope) {
// cast to our custom scope class
MyScope currentScope = (MyScope)scope;
// retrieve some records from the external server with an offset, for example
return fetchRecordsFromServer(currentScope.requestId, currentScope.offset);
}
public class MyScope {
private String requestId;
private Integer offset;
public MyScope(String requestId, Integer offset) {
this.requestId = requestId;
this.offset = offset;
}
}
Delay¶
The DELAY strategy allows you to pause for an arbitrary amount of time when starting up a Pull Link run. This is a very situational strategy but can be helpful for circumstances such as waiting for a file to be generated on an external server. Let’s say you call into the server during planFetch() and describe the records you want. The external server generates a CSV file somewhere and then exposes it to you. You could use the DELAY strategy to wait until that CSV file is ready to be read, check if the file is ready, and if it is you’d read the generated file with your fetchRecords() call.
You can delay as many times as you need to. You can leave the duration of the delay up to Valence (15 seconds ~ 90 seconds), or you can specify a number of minutes to wait as a minimum (handy if you know for sure that it’ll be a few minutes before things are ready).
If you use the DELAY strategy your Adapter must also implement the DelayedPlanningAdapter interface. This interface adds an additional method that Valence will call after the delay is over. You are welcome to return DELAY again from this method call, and keep doing that over and over until you are ready to move on to fetching records.
Definition¶
global static FetchStrategy delay(Integer minutes, Object scope);
global static FetchStrategy delay(Integer minutes, Object scope, Long expectedTotalRecords);
Example Usage¶
private String filePath = null;
public valence.FetchStrategy planFetch(valence.LinkContext context) {
// do some asynchronous operation that you know will take 5-10 minutes
String fileId = generateFile();
// ask Valence to call you back in 15 minutes
return valence.FetchStrategy.delay(15, fileId);
}
public valence.FetchStrategy planFetchAgain(valence.LinkContext context, Object scope) {
// get the state that we previously stashed in the scope object
String previousFileId = (String)scope;
// check to see if that file is ready yet
Boolean ready = checkFileStatus(previousFileId);
if(ready) {
filePath = getFilePath(previousFileId);
return valence.FetchStrategy.immediate(); // will call fetchRecords() in this same execution context with a null second parameter, which is why we set filePath
} else {
// wait two more minutes then try again
return valence.FetchStrategy.delay(2, previousFileId);
}
}
public List<valence.RecordInFlight> fetchRecords(valence.LinkContext context, Object scope) {
// retrieve and parse the file
List<valence.RecordInFlight> records = retrieveAndParse(filePath);
return records;
}
Tip
If you don’t want to specify a certain number of minutes to wait and instead let Valence decide, pass null as the first parameter: return valence.FetchStrategy.delay(null, myScope);
Locator¶
It’s unlikely you will ever need to use the LOCATOR strategy. This is a specialized strategy that uses a Database.QueryLocator to iterate over large quantities (millions) of records inside the local Salesforce org. Normally you are going to leverage the built-in Local Salesforce Adapter that comes with Valence for extracting records from the local Salesforce org. However, this strategy is available to you should you need to do this kind of thing yourself.
If you use the LOCATOR strategy your Adapter must also implement the SourceAdapterForSObjectPush interface. Valence will retrieve records using the locator and feed them to your sObject push method to process them. Your Adapter’s fetchRecords() is never called.
Definition¶
global static FetchStrategy queryLocator(String query);
global static FetchStrategy queryLocator(String query, Long expectedTotalRecords);
Example Usage¶
public valence.FetchStrategy planFetch(valence.LinkContext context) {
return valence.FetchStrategy.locator('SELECT Id, Name, Phone FROM Account');
}
public List<valence.RecordInFlight> buildRecords(valence.LinkContext context, List<sObject> records) {
// process sObject records in batches of context.batchSizeLimit or 2,000, whichever is smaller
}
No Records¶
Use the NO_RECORDS strategy if during planning you notice that you have no records that you need to fetch.
This will short-circuit the rest of processing; your fetchRecords() method will not be invoked, and the SyncEvent is immediately closed.
Definition¶
global static FetchStrategy noRecords();
Example Usage¶
public valence.FetchStrategy planFetch(valence.LinkContext context) {
Integer count = countRecordsToFetch();
return count > 0 ? valence.FetchStrategy.immediate() : valence.FetchStrategy.noRecords();
}
public List<valence.RecordInFlight> fetchRecords(valence.LinkContext context, Object scope) {
// retrieve some records
List<valence.RecordInFlight> records = goGetRecords();
return records;
}