Creating Your First Celigo Integration: Step-by-Step Tutorial

Start the process of integrating your business systems and data an be thrilling and intimidating. However, if you have access to Celigo, you can make this endeavour an easy and gratifying experience. In this Celigo step-by-step tutorial, we will walk you through the steps of developing your first Celigo integration. This tutorial will guide you through every step of the integration development process, from setting up the connectors to orchestrating data flow and automating critical business activities, whether you’re an IT specialist or a business user.

Our use case in this article will be synchronizing Salesforce accounts to Shopify customers. The idea is that we’ll read all account data from Salesforce that has been updated since our last integration run, and upsert those records as Shopify customers. We’ll focus on the build part here, then, in future articles, we’ll cover other topics such as troubleshooting, scheduling, production deployment, monitoring, etc. 

Creating an Integration

Before we create our flow, it’s better to create an integration which will act as a container for multiple flows. Remember, in Celigo’s terminology the integration process is called a flow, while an “integration” is nothing but a container for one or more flows. If we don’t explicitly create an “integration”, our flow will be placed under a built-in integration called “Standalone Flows”.

Let’s start by logging into the Celigo platform. We’ll be creating the artifacts under the “Sandbox” account so that in the future we can deploy our integrations to production, hence simulating a real-case scenario.

First step is to go on and create an “Integration” as shown in the below snapshot

Populate the integration name and optionally fill in a description, then click “Save & close”

You will be shown the “integration” that you have created with the list of current flows, which would be empty since we have not created any flow yet.

Creating a Flow

Next, we’ll create our flow which will be responsible for syncing Salesforce accounts to Shopify customers. To do that, hit “+ Create flow” as shown in the snapshot below

You will be presented with the flow canvas which is predominantly split into three sections, the Export which “exports” from a source, the Import, which “imports” data into a destination, and anything in between, such as adding a lookup, branches, hooks (custom code), etc. We have explained this more in our previous article Getting Started with Celigo: A Beginners Guide

Configuring the Export

To configure the Export, just click the “Add source” icon under the sources section, which is nothing but the “Exports” section, as shown below

Populate the fields as shown below and click “+ Create connection” to create a connection to Salesforce. We will be creating a connection since this would be the first time we are connecting to Salesforce. In the future, we can reuse the same connection in all of our flows when attempting to connect to Salesforce.

Populate the fields as shown below, then click “Save & authorize”. Once you do that, a popup will appear for you to login to Salesforce and authorize the connection

Once we do that, the “Create source” page will show the connection to Salesforce as online

Now that we have configured the connectivity to our source, click “Next” to navigate to the screen where we can create and configure our actual export.

In the Export screen, populate the fields as shown in the snapshot. Most important configuration fields to note are the Salesforce SOQL, where we can write a SQL-like query to extract data from Salesforce, and the Export Type where we can define if we want to export all data, or only changed data since our last export, or only new records for example. The question mark icon next to the field gives more information. For our use case, we want to export the delta, i.e, only changed records since the last modified date. For more information about SOQL and its syntax, check the Salesforce documentation site.

Also, note the Preview button which can be used to preview the data exported as per the SOQL query defined.

Once done, Save & close to save the changes and return back to the flow canvas

Configuring the Import

Next, let’s configure the Shopify connectivity and the Import. To do that, click the “Add destination / lookup” icon under the Destinations & Lookups section

Similar to how we configured the “export”, populate the fields as shown below and click “+ Create connection” to create a connection to Shopify. We will be creating a connection since this would be the first time we are connecting to Shopify. In the future, we can reuse the same connection in all of our flows when attempting to connect to Shopify.

Populate the fields as shown below, you will need to populate the Shopify store name, and the API key and secret. Follow the instructions on the Shopify documentation to get these. Once you do that, click “Test Connection”. Once you verify the connectivity is successful, click “Save & close” to configure the import. Note that you could use alternate methods for authentication such as OAuth2, but for the sake of simplicity, I opted for Basic auth in this example.

Next is to configure the Import. Follow the below snapshot to configure the import, however, note that we will change this configuration later. We’ll explain why in the next section

Configure a Salesforce Lookup

Recall when we configured the Salesforce accounts export in the first step, there was no email, first name, or last name among the exported fields. However, when we want to import to Shopify, these fields are mandatory. In fact, to determine if a record already exists in Shopify, we will rely on the email field. To retrieve those missing fields, we should create a lookup to export additional fields from the Salesforce contact object. To do that, click the plus sign before the Shopify import and select “Add destination / lookup”, as shown below

Configure the lookup as below. Note that we have selected the previously configured Salesforce connection. Click “Next” when done.

Populate the lookup details as shown below. Note that we are retrieving the additional fields from the “Contact” object now where the account id is equal to the incoming account id from the previous step. 

The nice thing is that you don’t have to memorize the query syntax. You can click on the “</>” icon to build and preview this query visually

Once you are done, click “Save & close” to save your changes and return back to the flow canvas.

Merge the Initial Data & the Lookup Data

Now that we have exported the data from Salesforce, and exported “additional” lookup data, we need to merge the two result sets so that we can use the merged results in the Shopify import step. To do that, click the results mapping icon after the lookup step as shown below

Map the three additional fields and preview to see the final output containing all needed fields, as shown below. Save & close when finished.

Map to Shopify Format

Now that we’ve got all required fields in our message, we need to map that into the format Shopify expects. To do that, we should use the “import mapping” icon as shown below.

Map incoming fields to Shopify fields as shown below. Preview to see how the record will look like in Shopify format. Save & close when done

Re-configure the Import

Now that we have the full data flowing through our flow with all needed fields, we can reconfigure the import to create or update records in Shopify based on the existence of the email field, which will act as the primary record identifier. To do that, open the import again by clicking the Import icon

Reconfigure the import as per the below snapshot. Note that we have selected “Create or Update” customer as the operation, and ran a dynamic search to identify existing records. 

Click “Launch” to open the configuration of the dynamic search and type email:{{{email}}} in the query field. This will send this string as part of the request sent to the Shopify API. 

Save & close to finish the configuration. Now we are ready to test the flow.

Salesforce & Shopping Status

Before testing, let’s have a look on the current data present in Salesforce

Let’s have a look on the current data present in Shopify as well

Test the Flow

To test the flow, we’ll first have to enable it, then hit the run button in the top menu. Once we do that, we’ll have to set the time to look for records. Since this is our first run, we’ll have to set it as custom and select a date which precedes any changes that have occurred on the data. After doing that, click “Run” to run the flow.

As you can see in the snapshot below, 13 records were extracted, however 11 were successful and 2 failed. Which is a good thing as this resembles what could most probably happen when you are building a flow, and gives us a chance to showcase some of the troubleshooting capabilities of Celigo.

Troubleshoot

Click on the errors label to navigate to the errors page. We can clearly see that one record had an invalid email address, and the other record had some missing fields.

We’ll tackle both errors differently, just to cover more scenarios. We’ll fix the invalid email in the source, i.e Salesforce, and we’ll filter any records with missing data in the Celigo flow. Let’s showcase the second case. 

Define an Input Filter 

Go back to the flow canvas and click the input filter icon just before the Shopify import. This allows us to filter which records we want to continue processing in later steps of the flow, and which records to drop.

In the input filter definition screen, set a rule that the email should not be empty. Preview to test your changes against the data in the input field

Save & close to return to the flow canvas.

Retest

Before retesting, just make sure to perform any update on both previously-failed records in Salesforce to ensure they will be picked up in the new flow run. After doing that, we are ready for our second run attempt. Hit run again and this time keep the Delta flow config as “Automatic”

You should be able to see now that two records have been exported, one which made its way into Shopify successfully, and the other was skipped because it lacked the email address which is mandatory in Shopify

Note: If you are new to Celigo, don’t mind the number of errors which still shows under the Shopify import. Celigo shows all previous errors that occurred on each step. So this is something you can safely ignore.
Finally, navigate to Shopify and view all newly created customers

Conclusion

Congratulations on completing your first Celigo integration! In this step-by-step tutorial, we have created our first flow in Celigo which integrates Salesforce and Shopify. We have only covered the build and test aspect. We’ll cover other concerns in future articles, such as deployment, monitoring, detailed troubleshooting and error handling. Beware, the journey of integration in Celigo has just begun. So, let’s jump on this exciting path of innovation and continue to thrive in an increasingly connected business landscape with Celigo by our side!