Operating DataHub
This section describes how you can access system information, usage statistics, and audit logs.
Checking system information
Info: You need administration permissions to access system information. See section Defining DataHub permissions and roles for details.
In the navigator, select Administration and then System status to get information about the system configuration and its status.
Under Microservice you will find the status of the microservice, which is either marked as green or red. This status reflects whether the microservice can be accessed from the web application. If the microservice is accessible, its current version is shown. If not, check the status of the microservice and its logs as described in section Managing applications.
Under Web application you will find the version of the web application.
Under Management you will find the setup of the system. If you expand that box by clicking on the arrow to the right, all relevant system properties and their values are listed. Note that these values cannot be modified for a running microservice. The tenant administrator needs to redeploy the microservice with corresponding new values.
Tracking usage statistics
If enabled, DataHub tracks usage statistics on the amount of data being processed. These statistics are collected for the offloading queries and track the amount of data these queries read from the Operational Store of Cumulocity IoT. The statistics are also collected for the ad-hoc queries and track the amount of data these queries read from the data lake. The usage statistics can be utilized for a volume-based charging. They can also be utilized to pinpoint resource-intensive queries in terms of network load.
Info: The tracking of usage statistics is supported for the DataHub Cloud edition. It is not supported for the DataHub Edge edition.
In the navigator, select Administration and then Usage statistics to view the usage statistics.
In the action bar, a date control allows you to select the month for which you want to see the usage statistics.
The three top panels show overall summary statistics as well as statistics separated for offloading and ad-hoc queries. If data from the month before the selected month is available, a tendency arrow illustrates whether the data volume of the selected month has decreased, increased, or stayed flat. The panels with the offloading and the ad-hoc query statistics additionally list the days with minimum/maximum volume as well as the daily average volume.
The table below the summary statistics shows the details on a per-day basis for the selected month. For each day, the volume offloaded and the volume queried are shown as well as their sum, which constitutes the daily volume. In addition the percentage of the monthly volume is shown, i.e., how much did the daily volume contribute to the overall monthly volume. The date of each entry links to the Query log, which lists all queries for the respective day.
Info: The statistics are refreshed once per hour. Therefore, the statistics for the current month may not include the latest data. The statistics are deleted after a retention period, so for older months statistics may no longer be available.
Viewing audit logs
Auditing shows in the query log the queries being executed and in the system log the operations that users have carried out.
Query log
In the navigator, select Auditing and then Query log to view the query log.
Info: The DataHub feature for storing query profiles needs to be enabled. The profiles are deleted after a retention period, so for older months profiles may no longer be available.
In the action bar you can choose between offloading queries and ad-hoc queries, define a text filter on the offloading task/ad-hoc query string, and select a time period. Click Apply to get the logs with the respective settings. Use the pagination controls at the bottom of the page to navigate through the result list.
For each offloading query, the following information is provided:
| Column name | Description |
|---|---|
| Offloading task | The task name of the offloading pipeline, complemented by a status icon showing success or failure of the pipeline execution |
| Runtime (s) | The runtime of the execution in seconds |
| Data scanned (MB) | The amount of data the offloading query has read from the Operational Store of Cumulocity IoT |
| Data billed (MB) | The amount of data being billed; amounts of data less than 10 MB in an offloading query will be billed as if they were 10 MB |
| Details | The internal task ID in an expandable box |
For each ad-hoc query, the following information is provided:
| Column name | Description |
|---|---|
| Query | The SQL query, complemented by a status icon showing success or failure of the query execution |
| Runtime (s) | The runtime of the execution in seconds |
| Data scanned (MB) | The amount of data the ad-hoc query has read from the data lake |
| Data billed (MB) | The amount of data being billed; amounts of data less than 10 MB in an ad-hoc query will be billed as if they were 10 MB |
| Details | The link to the the internal Dremio job in an expandable box |
System log
In the navigator, select Auditing and then System log to view the system log.
The audit log shows the last 10 logs. In the action bar, you can change that number by setting a new limit and refreshing the audit log. Additionally you can filter the entries by text or status by using the filter controls in the action bar.
For each log entry, the following information is provided:
| Column name | Description |
|---|---|
| User | The user that has carried out the operation |
| Event | The type of operation |
| Details | The details of the operation and, if available, further information in an expandable box |