File upload API

Sending us your customer data via our File Upload API can be easier and more flexible than SFTP or manual file upload. Here we’ll guide you to creating your API token, requesting permission to deliver a file, and sending that file.

There are other options for file transfer that might be worth considering:

• Providing JSON data to Lexer
• Upload using SFTP
• Upload using S3

Its always a good idea to discuss with your Success manager if you are unsure which method would work best.

How do I upload files?

Sending your file is done in three steps.

1. Create an API token.
2. Authenticate and request permission to send your file. This will generate a destination URL.
3. Use this destination URL to send us your data file.

Once the data has been sent you will be able to view it in Datasets. Once you have a dataset

1. Create an API Token

For more detailed instructions of this process have a look at our Authentication and API token creation article.

Please note:

• Your account can have up to 5 active API tokens.
• Certain rate limits are applied to API keys to manage traffic efficiently. It may be increased depending on your use cases. Please your Success Manager to discuss it further. Error 429 will indicate an exceeded rate limit.
• All APIs have the option for IP Whitelisting as a secondary layer of security, some APIs, like Profile Read API, have a mandatory requirement for an IP Whitelist.

2. Authenticate and request permission to send your file

To successfully authenticate and run your file transfer you will need to use the following command, replacing the API token and File name.

Run the following command after replacing the [.code]API_token[.code] with your API code (created in step 1). Replace the [.code]File_Name[.code] section with the name of your .csv or .json file (make sure to include the appropriate suffix eg. .json).

curl -XPOST -H 'Auth-Api-Token: API_token' https://api.lexer.io/v1/uploads -H 'Content-type: application/json' -d '{"filename": "File_Name"}'

For example, when using a JSON file you would use the following command:

 # Request
curl -XPOST -H 'Auth-Api-Token: 12345678-1234-1234-1234-123456789' https://api.lexer.io/v1/uploads -H 'Content-type: application/json' -d '{"filename": "{specification_name}-{date}-{version}.json"}'

A successful request will return a response that looks like this:

# Response
{ id: 123, url: "https://lexer-client-upload-au..." }

The URL field created is where you will be sending your file. Copy the URL for use in the next step.

3. Send your data file

To send your file, run the command below with the [.code]URL[.code] inserted from the response above and your [.code]File_name[.code].

curl -XPUT "URL" --upload-file File_Name

For example, when using a JSON file you would use the following:

 curl -XPUT "https://lexer-client-upload-au..." --upload-file {specification_name}- {date}-{version}.json 

Done! Our team will receive a notification that your file has been received, and we will reach out with any questions or issues. We advise getting in touch to let us know you have sent the file.

Troubleshooting

Please note that the URL received from first request contains [.code]\u0026[.code] in place of [.code]&[.code]. Having the ampersand represented as hexadecimal can cause the second request to return an error similar to that below.

<?xml version="1.0" encoding="UTF-8"?><Error><Code>AuthorizationQueryParametersError</Code><Message>X-Amz-Algorithm only supports "AWS4-HMAC-SHA256"</Message><RequestId>...</RequestId><HostId>...+.../....=</HostId></Error>

This can be avoided by saving the URL in a variable as demonstrated in the workflow below.

url=$(curl -XPOST -H 'Auth-Api-Token:<TOKEN>' https://api.lexer.io/v1/uploads -H 'Content-type: application/json' -d '{"filename": "<FILE_NAME>"}' | jq -r .url)
curl -XPUT "${url}" --upload-file <FILE_NAME>

Python Implementation

Below is a python implementation of the approach to uploading files with the File Upload API.

#!/usr/bin/env python3
import requests
TOKEN = "<TOKEN>"
FILE_PATH = "<FILE_NAME>"

def request_presigned_url(token, file_path):
    url = "https://api.lexer.io/v1/uploads"
    payload = {"filename": file_path}
    headers = {"Auth-Api-Token": token}
    response = requests.request("POST", url, json=payload, headers=headers)
    return response.json()
    
def upload_file(url, file_path):
    response = requests.put(url, data=open(file_path, "r").read())
    return response.status_code
    
presigned_url = request_presigned_url(TOKEN, FILE_PATH)
upload_status = upload_file(presigned_url["url"], FILE_PATH)
print(upload_status)

This is the simplest example. For more detail, you can look at our repository here.

That's a wrap

That's it for uploading API files! If you get stuck, try following the steps above and please don't hesitate to reach out to Lexer Support at support@lexer.io if you have any questions.