Managing Fusion's System Metrics Collection

Managing the system_metrics Collection

As close to 600 metrics are indexed to the system_metrics collection at regular intervals, the data can grow to be quite large over time. Older data can be deleted accordingly to manage the size of the metrics collection.

To delete the metrics data that is older than a month, the below query can be used,

http://localhost:8983/solr/system_metrics/update?stream.body=<delete><query>timestamp_tdt:[* TO NOW-1MONTH/MONTH]</query></delete>&commit=true

To delete all the data in the system_metrics collection (use with caution),

curl http://host:port/solr/system_metrics/update?commit=true -H 'Content-type:application/json' -d '{"delete": { "query":"*:*" }}'

Scheduling the Metrics Cleanup Job

The query below can be used to check the current configuration of the metrics cleanup scheduler job,

curl -u user:pass http://localhost:8764/api/apollo/scheduler/schedules/delete-old-metrics

{
   "id": "delete-old-metrics",
   "creatorType": "system",
   "creatorId": "MetricSchedulesRegistrar",
   "createTime": "2017-03-02T04:57:13.257Z",
   "startTime": "2017-03-02T04:57:13.257",
   "repeatUnit": "WEEK",
   "interval": 1,
   "active": true,
   "callParams": {
       "uri": "solr://system_metrics/update",
       "method": "GET",
       "queryParams": {
           "wt": "json",
           "stream.body": "<delete><query>timestamp_tdt:[* TO NOW-13MONTHS]</query></delete>"
       },
       "headers": {}
   }
}

In the “queryParams” section in the above output, we can see that the metrics older than 13 months are deleted ( timestamp_tdt:[* TO NOW-13MONTHS] )

To modify the cleanup duration from 13 months to 1 month, use the query below,

curl -u user:pass -X PUT -H 'Content-Type: application/json' -d '{
   "id": "delete-old-metrics",
   "creatorType": "system",
   "creatorId": "MetricSchedulesRegistrar",
   "createTime": "2017-03-02T04:57:13.257Z",
   "startTime": "2017-03-02T04:57:13.257",
   "repeatUnit": "WEEK",
   "interval": 1,
   "active": true,
   "callParams": {
       "uri": "solr://system_metrics/update",
       "method": "GET",
       "queryParams": {
           "wt": "json",
           "stream.body": "<delete><query>timestamp_tdt:[* TO NOW-1MONTH]</query></delete>"
       },
       "headers": {}
   }
}' http://localhost:8764/api/apollo/scheduler/schedules/delete-old-metrics

Metrics Collection Frequency

The default metrics collection frequency is 60 seconds, which can be changed by modifying the com.lucidworks.apollo.metrics.poll.seconds configuration parameter using the Fusion Configurations API.

For example to set com.lucidworks.apollo.metrics.poll.seconds to 600 seconds,

curl -u user:pass -X PUT -H 'Content-type: application/json' -d '600' http://localhost:8764/api/apollo/configurations/com.lucidworks.apollo.metrics.poll.seconds

 

Disabling Metrics

Metrics Collection can be disabled by setting com.lucidworks.apollo.metrics.poll.seconds to 600 seconds to “-1”, as shown below,

curl -u user:pass -X PUT -H 'Content-type: application/json' -d '-1' http://localhost:8764/api/apollo/configurations/com.lucidworks.apollo.metrics.poll.seconds

Have more questions? Submit a request

0 Comments

Please sign in to leave a comment.
Powered by Zendesk