Adventures in Salesforce and DataWeave (Beta) Part 1

Every so often a new development in the world of Apex promises to be a game changer, and perhaps DataWeave offers the best chance of achieving that goal. For those of us used to doing integrations between different systems and needing to do some transformations or translations from say XML to json then this will potentially be a game changer – provided that the performance is up to it (and initial indications are not great).

So lets create a scratch org to do some work with:

sfdx force project create --projectname DataWeaveTesting

This will create the scaffolding for a new project in a new directory called DataWeaveTesting. Our project-scratch-def.json file which is in the config folder needs to be updated to include the new DataWeave feature.

{
  "orgName": "Demo company",
  "edition": "Developer",
  "features": ["EnableSetPasswordInApi","DataWeaveInApex"],
  "settings": {
    "lightningExperienceSettings": {
      "enableS1DesktopEnabled": true
    },
    "mobileSettings": {
      "enableS1EncryptedStoragePref2": false
    }
  }
}

Now create a new scratch org and do a first blank push

sfdx force org create --setdefaultusername --setalias DataWeaveScratch --definitionfile config/project-scratch-def.json -d 5 -s

Lets have a look at an example. This website https://www.appsloveworld.com/free-online-sample-xml-api-for-testing-purpose has a free XML API that we can use.

So https://restapi.adequateshop.com/api/Traveler?page=1 gives an XML feed of data and we will read this in, transform it and then output as a JSON file. So the top of the file looks like this:

<TravelerinformationResponse xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<page>1</page>
<per_page>10</per_page>
<totalrecord>17091</totalrecord>
<total_pages>1710</total_pages>
<travelers>
  <Travelerinformation>
    <id>11133</id>
    <name>Developer</name>
    <email>Developer12@gmail.com</email>
    <adderes>USA</adderes>
    <createdat>0001-01-01T00:00:00</createdat>
  </Travelerinformation>
  <Travelerinformation>
    <id>11134</id>
    <name>AS</name>
    <email>qweqw@mail.ru</email>
    <adderes>USA</adderes>
    <createdat>0001-01-01T00:00:00</createdat>
  </Travelerinformation>
..
..
..
</travelers>
</TravelerinformationResponse>

Create an Apex class and let’s get the Web call in a method that we can check:

public class DataWeaveXmlToJSON {
  public static string CallWebSite() {
    Http http = new Http();
    HttpRequest request = new HttpRequest();
    request.setEndpoint('http://restapi.adequateshop.com/api/Traveler?page=1');
    request.setMethod('GET');
    HttpResponse response = http.send(request);
    string returnResults = '';
    if (response.getStatusCode() == 200) {
      returnResults = response.getBody();
    } else {
      returnResults = 'Error getting data';
    }
    return returnResults;
  }
}

You will need to add http://restapi.adequateshop.com to your Remote Site Settings.

We can now test that we get an XML file in by entering this in the Developer Console and the Debug Window:

system.debug(DataWeaveXmlToJSON.CallWebSite());

So at this point we can actually start to look at the way DataWeave actually works.

There are some extensive tutorials on Dataweave and a playground to test things.

• https://dataweave.mulesoft.com/learn/tutorial is the main tutorial area
• https://dataweave.mulesoft.com/learn/playground can be used to test out (and then move across to Apex)
• https://docs.mulesoft.com/dataweave/2.4/ is the link to the current docs

We have two main aims for this post:

• Convert the file to json
• Change some of the field names as they are not spelt correctly

DataWeave in APEX really consists of only two main items, a .dwl file which contains our transformation file and then some apex code to push data into the .dwl transformation file. So these two lines will be pretty common in DataWeave projects:

 Dataweave.Script dwscript = DataWeave.Script.createScript('transformation_name');

 DataWeave.Result result = dwscript.execute(new Map<String, Object>{ 'data' => xml });

The first line loads the transformation file in and then the second line executes the DataWeave transformation. We pass in a Map of the data files with a string description – in the case above we have a data description (payload name) and an xml string. We can then read the results of the transformation with :

string jsonResult = result.getValueAsString();

dwl file

%dw 2.0
input data application/xml
output application/json

---
{
"details": data.TravelerinformationResponse.travelers.*Travelerinformation  map (item, index) -> {
      id: item.id,
      name: item.name,
      email: item.email,
      address: item.adderes,
      createddate: item.createdat
}
}

Lets break this down in to the various bits:

• The %dw 2.0 is the header of the file and gives the version of DataWeave being used
• The input line is say that we should expect a xml file to be passed in to an object called data (other examples use payload as the name)
• The output line says that we should provide the output in json format
• The — signifies the start of the transformation (in more advanced examples we can create functions before this —)
• Below the — we actually define how we want to transform the data object we defined in the input

When we look at the structure of the main area of the file, we see that we are creating a JSON object with the { } and we have an initial JSON object called details.

{
"details": [Array of entries will go here]
}

Our xml file is based on a hierarchy of nodes and our data object starts off with a node called ‘TravelerinformationResponse’ which as a node called ‘travelers’ and then a list of ‘Travelerinformation’ nodes.

<TravelerinformationResponse>
..
<travelers>
  <Travelerinformation>

In this case we want to transform these individual nodes into an array of json objects. So we can use the * as a wildcard to provide an array of the matching nodes. So the statement below says ‘provide an array of the all the objects in our data xml object called Travelerinformation as long as they are inside a node called travelers which needs to be inside a node called TravelerinformationResponse and part of the data input xml file :

data.TravelerinformationResponse.travelers.*Travelerinformation

In many ways DataWeave is like javascript and has similar commands and syntax. So we can use a map command to pass through the array of items and then iterate through them and output a new json array. So this takes the form of :

data.TravelerinformationResponse.travelers.*Travelerinformation  map (item, index) -> {
      id: item.id,
      name: item.name,
      email: item.email,
      address: item.adderes,
      createddate: item.createdat
}

Each ‘item’ in the map is one of the Travelerinformation objects and it will have the structure:

<id>11133</id>
<name>Developer</name>
<email>Developer12@gmail.com</email>
<adderes>USA</adderes>
<createdat>0001-01-01T00:00:00</createdat>

So we can use item.id to pick up the id of the record. Some of our xml is incorrectly spelt, so we can say use

address: item.adderes

to transform the data into a new heading. So this code will translate the XML above into a JSON extract like this:

"id": 11133,
"name": Developer,
"email": Developer12@gmail.com,
"address": USA,
"createddate" 0001-01-01T00:00:00

In Part 2 we will look at how we do this in Apex.