Providing JSON data to Lexer

Short of using one of our integrations, APIs or exports of data via S3/SFTP in JSON format are the fastest ways to deploy the Lexer CDXP within your organization.

Steps to provide JSON data to Lexer

1. Determine the schema best suited to your data.
2. Export and transform the data into the provided format.
3. Determine the process for delivering the data.
   [.in]a. APIs: Read Getting started onboarding data to figure out the best solution for you.[.in]
   [.in]b. SFTP/S3: Upload using SFTP or Upload using S3.[.in]
   [.in]c. File uploader: Read File Upload API for specifics.[.in]
   [.in]d. Custom: Chat with your Lexer contact.[.in]
4. Download and validate the data prior to uploading.
5. Transfer the data.
6. Notify your Lexer contact.

File Naming

Files must be named using the following format: 

[.code]{specification_name}-{date}-{version}.json[.code]

This format consists of the following three parts:

1. Specification Name: A descriptive name for the files contents, separated by an underscore `_`.
2. Date: An ISO8601 date for the file, reflecting the contents or date of file generation.
3. Version: An optional version identifier. We recommend using versions. This allows you to easily revert to an old file if you identify any issues with the current version.

File names are case-sensitive, and may not contain special characters other than a hyphen '-', or an underscore '_'.

File Naming Examples

Let’s look at an example for commerce returns made on August 29th, 2022:

• [.code]commerce_purchases-20220829.json[.code]

In the case where a file might need to be overwritten with a newer version, you can add an additional suffix. This will update the file currently being used, while keeping the old file.
like so:

• [.code]commerce_purchases-20220829-a.json[.code]

• [.code]commerce_purchases-20220829-b.json[.code]

The higher suffix will cause the newer version to be used in place of previous versions. In these examples [.code]commerce_purchases-20220829-a.json[.code] will be ignored, and [.code]commerce_purchases-20220829-b.json[.code] used.

Alternatively, you could simply delete the [.code]commerce_purchases-20220829-a.json[.code] file entirely, although this is not the recommended option. 

Validating JSON data 

Lexer provides JSON schema files for each of the data specifications supported by the CDXP. You can download copies of each schema from our Github repository.

Profile schema:

• [.in]Customer record[.in]‍

Commerce schema:

• [.in]Product record[.in]
• [.in]Purchase event[.in]
• [.in]Return event[.in]

Marketing schema:

• [.in]Email subscribe event[.in]
• [.in]Email send event[.in]
• [.in]Email open event[.in]
• [.in]Email click event[.in]‍
• [.in]Email bounce event[.in]‍
• [.in]SMS subscribe event[.in]‍
• [.in]SMS send event[.in]‍
• [.in]SMS click event[.in]

Once downloaded, you can use our File Upload API to validate or validate and upload your file (the links provide more detail about how this can be done). Before uploading you will need an API token, you can learn more about how to generate a token in the Hub here.

To validate:‍

./upload.py validate \
    --local-filename my_file.ndjson \
    --record-type customer_record

‍To validate and upload:

./upload.py upload_validate \
     --local-filename my_file.ndjson \
     --record-type customer_record \
     --destination-dataset-id a1b2c3d4 \
     --api-token xxxxxxx

This script raises an error if your file doesn't comply with the schema, allowing you to address data quality issues prior to transferring. This helps your team troubleshoot data quality issues and reduces back and forth with the Lexer team.

Need some help?

This step can be a little tricky. There are a number of errors you could run into, so please reach out to your Success Manager if you need a hand and we can help you to troubleshoot.