Clean Fusion System Metrics and Update System Metrics Scheduler Job

In order to clean the system_metrics index that is older than a month— that which is already created since few months—use the below delete query

http://localhost:8983/solr/system_metrics/update?stream.body=<delete><query>timestamp_tdt:[* TO NOW-1MONTH/MONTH]</query></delete>&commit=true

To do a full clean of system_metrics index, use the below query

curl http://host:port/solr/system_metrics/update?commit=true -H 'Content-type:application/json' -d '{"delete": { "query":"*:*" }}'

Further, in order to reset the current scheduler job—"which is just deleting the metrics logs that are older than a year"— to make it clean the metrics logs older than a month, Follow the below steps. 1) 

Using the below curl command, query for json response of scheduler.
curl -u admin:admin@123 http://localhost:8764/api/apollo/scheduler/schedules/delete-old-metrics
Body response will be as shown below

{
    "id": "delete-old-metrics",
    "creatorType": "system",
    "creatorId": "MetricSchedulesRegistrar",
    "createTime": "2016-01-20T07:43:51.957Z",
    "startTime": "2016-01-20T07:43:51.957Z",
    "repeatUnit": "WEEK",
    "interval": 1,
    "active": true,
    "callParams": {
        "uri": "solr://system_metrics/update",
        "method": "GET",
        "queryParams": {
            "wt": "json",
            "stream.body": "<delete><query>timestamp_tdt:[* TO NOW-13MONTHS]</query></delete>"
        },
        "headers": {}
    }
}

2) Right now, as per the above configs, scheduler is deleting metrics logs that are older than an year. So, You need to update this if you just want to retain logs of a month and delete the rest. To do that,

  1. Copy the entire response json
  2. Just edit the delete query part to : <delete><query>timestamp_tdt:[* TO NOW-1MONTH]</query></delete>
  3. Further, post it back to fusion using 'put' curl method as shown below[example curl]. (I used postman to post this curl command. If you are not comfortable with command line, use postman. There is a blog here to help you get started with postman[1])

[example curl] Curl command to post back the system metrics logging frequency changes

curl -u user:pass -X PUT -H 'Content-Type: application/json' -d '{
    "id": "delete-old-metrics",
    "creatorType": "system",
    "creatorId": "MetricSchedulesRegistrar",
    "createTime": "2016-01-20T07:43:51.957Z",
    "startTime": "2016-01-20T07:43:51.957Z",
    "repeatUnit": "WEEK",
    "interval": 1,
    "active": true,
    "callParams": {
        "uri": "solr://system_metrics/update",
        "method": "GET",
        "queryParams": {
            "wt": "json",
            "stream.body": "<delete><query>timestamp_tdt:[* TO NOW-1MONTH]</query></delete>"
        },
        "headers": {}
    }
}' http://localhost:8764/api/apollo/scheduler/schedules/delete-old-metrics

 

If you would need the system metric clean to happen every week, use the below curl command.

curl -u admin:password123 -X POST -H "Content-Type: application/json" 'http://localhost:8764/api/apollo/scheduler/schedules' -d '
{
  "id" : "delete-old-metrics",
  "creatorType" : "human",
  "creatorId" : "DefaultSystemLogCleanupScheduleRegistrar",
  "createTime" : "2015-08-13T08:32:21.916Z",
  "startTime" : "2015-08-13T08:32:21.916Z",
  "repeatUnit" : "WEEK",
  "interval" : 1,
  "active" : true,
  "callParams" : {
    "uri" : "solr://system_metrics/update",
    "method" : "GET",
    "queryParams" : {
      "stream.body" : "<delete><query>timestamp_tdt:[* TO NOW-7DAYS]</query></delete>",
      "wt" : "json"
    },
    "headers" : { }
  }
}'

[1] https://support.lucidworks.com/hc/en-us/articles/213097778-POSTMAN-Extension-for-CURL-Commands-or-RESTFUL-EndPoints-

 

Let me know if you get into any issues.

 

PS:-The updated knowledge base article is available at 

https://support.lucidworks.com/hc/en-us/articles/115004006287-Managing-Fusion-s-System-Metrics-Collection

 

 


 

Have more questions? Submit a request

0 Comments

Please sign in to leave a comment.
Powered by Zendesk