Monitoring your data in the Hub has just got even easier with Logs. Logs lets you view your data load and processing logs status at a glance. Logs aren't just about monitoring, they can also help troubleshoot data issues, making the resolution with the Lexer team even quicker. Get ready to take control of your data management like never before!

Understanding Logs components

Let’s start by navigating to Logs, navigate to Manage > Logs in your Hub. 

There are 6 components that Logs shows in the main page, you’ll also see 4 filters on the top of the page to help you find what you need. 

1. Job type

The specific type of job. A description of job types can be found below. 

2. Name 

The name of the job. 

3. Status

The current status of the job, there are 4 potential statuses:

4. Source

The source indicates where the job originated from and is inferred from the data that’s received. ‍

5. Created at

The time the job began. 

6. Duration

The total time taken for the job to complete. 

Navigating and filtering Logs

There are a few options to help you refine your search to find the correct logs. 

You can start by selecting which column will dictate the order. To do this, click on the headers for any of these columns to filter by ascending or descending order which is indicated by a corresponding arrow icon. 

The filters located at the top of the page enable you to further refine your search based on the components outlined in the previous section. When adding filtering options, you will see a count in brackets showing the number of selections within that filter. 

Last 7 Days - A date picker for the time period you’d like to search which defaults to the last 7 days.

Job Type - Select as many job types as you like from the drop down menu. 

Status - Select the types of status you are interested in seeing in your search. 

Dataset - Refers to the specific dataset that relates to the job. The name of the dataset will contain the dataset ID and dataset name ie. [.code]datasetID - datasetname[.code].

Once you make your selections in the filters, click on the apply button on the right side of the page. You will see the number of results adjusted to show the total number of jobs found in the search. 

Health checking your data

The best way to make sure your data is accurate is to conduct regular health checks. Within Logs you can see all of the load jobs, the process of ingestion of data from start to finish. It allows you to determine how your CDP is going and whether everything has been ingested properly. Datasets is a bit more granular, allowing you to check on the health of a dataset specifically, for more information about Datasets you can check out our Learn articles here. 

Finding a job or group of jobs

If you are attempting to find a particular job, use a combination of filtering to narrow down the scope. For example you might want to check on dataflows in the last 30 days. 

1. Add a 30 day time frame to the date picker by clicking on “Last 7 Days”. 
2. Select dataflows in the “Job type” field. 
3. Click Apply on the right to initiate the search. 

You’ll now be able to see all dataflows in the last 30 days. You can narrow this down even further by adding a status filter if you are trying to identify any failed or pending dataflows. 

More detail with the job view

If you are looking for some additional information about a job, you can click on the job. This will open up the Job View. 

The job view adjusts based on the type of job as some details are not universal. 

Each view will contain:

• Status
• Created at
• Updated at
• Job Type
• An ID (format will vary depending on job type)
• Name
• Duration (if relevant)
• Provider (integrations only)
• Pipeline type (integrations only)

This additional information can be really helpful if you are worried some data is yet to arrive into the Hub. You can check, for instance, the pipeline type to determine whether an integration is scheduled. 

Need some help?

If something looks a bit off and you need some help, you can copy the exact job you are having an issue with and send this to your Success Manager or Support. To share, open the job view, copy the URL and send it through, it's that simple. This provides us with more information about the search query you have used, the job that needs some attention, and ultimately speeds up the investigation significantly. 

That’s a wrap!

In this article we covered how you can assess the status of data load and processing logs, making sure everything is working as it should, and how to use Logs to troubleshoot your data issues. Logs serves as an invaluable tool for troubleshooting data issues, speeding up resolutions with the dedicated support of the Lexer team. Logs is here to add that extra oomph to your data game! 🚀