Boomi: Recursive Connector Call to Handle Pagination

Hello everyone! In this blog, I’m going to show you how to do recursive connector call in Boomi to handle pagination.

 

This is useful when you have a source system that returns the result in a batch with a fixed number of records in a single API call instead of all records. For example, the source system only returns the first 100 records out of 2000 records in the first API call. In order to get the next batch (or next page of result), you have to do another API call. Sounds easy right?! But, how can you achieve this in Boomi?

 

There are few steps to implement the logic:

1. create an incremental counter (increase by 100 or more – depends on the batch size of the source system)
2. create a property to hold the number of total record
3. use a decision shape to compare the counter with the number of total record to find out whether there are more records to query
4. connect the last shape to the connector shape to do the next call if necessary

 

 

Figure 1 illustrates the recursive logic in the process

 

When you execute the integration process, you should see something similar shown in Figure 2. As you can see in the process logs, there are 1239 records inside the source system. The HTTP connector will query 100 records in a batch until the last batch.