Batch Processing is the key to building a high-performing integration with Scribe Online – but there’s a little more to batch than enabling the check box!
When you enable batch processing, you’re asking Scribe Online to gather a group of source rows and send that data to the target together [as a batch]. Batching dramatically increases performance when Scribe Online does not need to go back-and-forth between source and target, especially for cloud endpoints. Be aware that putting lookups in a map will decrease your performance, even when you have batch processing enabled. This flow chart gives an overview of what goes on behind the scenes when you are processing data in batch mode.
In this podcast, Mark and I will discuss how Scribe batches together source rows, when and where the transformations happen, as well as tips for getting the most out of batch processing. Additionally, I will walk-thru designing transactional vs. migration maps, offer tips for removing target Lookups, and caching data locally to get the most out of batch processing.