Posts Tagged ‘talend’

Talend Tips: Processing Large Data when Using tUniqRow

In Talend, tUniqRow is a useful component which allows you to filter out only the distinct unique row from a set of data. While using this, you may hit into Java Heap Space issue (java.lang.OutOfMemoryError) if the data to be processed is very large (millions of rows).   One possible solution to this is by…

Read More

Talend tSalesforceOutputBulkExec Component

Salesforce supports cascading update and inserts function by using reference key. In Talend, we can perform the cascading UPSERT function by using tSalesforceOutputBulkExec or tSalesforceOutput components. The benefit of using this function is to reduce the API call from Talend to Salesforce. Imagine the number of calls to make if there are millions of record for data migration.In order…

Read More

Talend tContextLoad

In Talend, you can use the tContextLoad to load the value from a properties file to the context variables. The concept is the same with Java properties file. It is extremely handy when it comes to the following situation: Username and password Configurable flag File path to read and write data   Below is an…

Read More

Talend: How to use tFileOutputMSDelimited (multi-schema output)

This is a simple guide to show how you can use Talend to generate a complex delimited file (e.g, combination of parent and related child records) by using the tFileOutputMSDelimited component. In this scenario, the sample output that we will be generated is:   Parent Row 1,P1,This is sample description,2013-08-01 Child Row 1,C1,P1 Child Row…

Read More

Talend Tips: Trigger a Job using Socket Listener

We talked about triggering a Talend job by using file listener previously. Today, let’s proceed to the next component which is the tWaitForSocket.   This component utilizes the ServerSocket object in Java to act as a server to listen to a port and trigger the integration process if there is an activity. As you can…

Read More

Talend Tip: Validating XML file with XML Schema (XSD)

All of us know that generating XML data in the integration process is easy, however, generating an XML file that is compliant with the standard of a target application is hard as the data from the source application may be different from the target application. The best practice to prevent sending invalid XML data to…

Read More

Talend Tips: Returning data from Child to Parent process

Sometimes, I prefer to split a very large and complicated logic process into multiple sub-processes and have them executed in sequence. This way, I can manage the process better and make my testing as well as troubleshooting easier. Ever wonder how does the data in such organized process being transmitted from one sub-process to another?…

Read More

Talend Tips: Trigger a Job using File Listener

In Talend, there are 3 useful listener components that we can use to implement an integration process that is being executed/triggered on-demand. The first listener component that we discuss here is tWaitForFile. This component allows you to listen on a file directory and execute an integration process whenever a file is dropped into the directory,…

Read More

Talend: use tSchemaComplianceCheck to validate data

Inserting data from a system to another could be quite troublesome at times especially due to the difference in field length for both systems. However, this can be easily overcome in Talend by utilizing the tSchemaComplianceCheck component. This component can help to validate the data according to the schema that you have defined and it…

Read More

Talend Tips: User Defined Map Function

We have discussed a lot of Talend Integration Studio’s component previously. Today, let’s move on to another interesting tip where we will define our own custom function and use it as a map (User Defined Map Function). In order to create a user-defined map function, first, you will need to create a custom Routine in…

Read More