PM (Predictive Merchandiser) rules fail to load on Fusion UI

Goal

Prior to Fusion 5.9.2 the PM rules page stops loading as the GET call to targeted-docs becomes huge and hits the hard limit eventually makes the Rules UI un usable. This documents helps in achieving a solution for such a scenario by pushing back all offending rules to next page and reducing the length of the GET call.

Environment

Fusion 5.x until Fusion 5.9.2

Steps

1. Initial step should be to collect the output of (HAR file or the network output from browser)
2. The HAR file should provide details on whether the call to targeted-docs, which is triggered on page load, is causing the issue.
3. In the above snapshot, the request to targeted-docs, an internal GET request, is passing a list of document IDs as a URL parameter. The length of this parameter exceeded approximately 6,000 characters, causing the UI to fail to load the rules. Sample URL:

   https://exampledomain.com/api/apps/flash_uki_app/query-rewrite/search/targeted-docs?docId=21L50014UKIEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,21HM0077UKIEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,21F8CTO1WWIE2IEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,21HMCTO1WWIE2IEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,21JNCTO1WWIE2IEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,21HQCTO1WWIE3IEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,21KC005RUKIEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,21MS0047UKIEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,21M7002MUKIEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,21MA001VUKIEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,21FACTO1WWIE2IEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,21KGCTO1WWIE2IEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,21KHCTO1WWIE3IEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,21J8CTO1WWIE2IEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,M10PLUS3UK2IEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,ZAAJ0385GBIEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,ZAD00071GBIEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,ZACH0200GBIEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,83DE0002UKIEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,83DECTO1WWIE3IEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,83DHCTO1WWIE2IEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,82Y5CTO1WWIE2IEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,83DG0046UKIEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,83DGCTO1WWIE1IEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,83DFCTO1WWIE2IEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,90UY002VUKIEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,83BGCTO1WWIE1IEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,83DA001UUKIEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,83DA001VUKIEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,12E3CTO1WWIE2IEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,12EHCTO1WWIE2IEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,12HKCTO1WWIE2IEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,12E9CTO1WWIE2IEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,12DTCTO1WWIE2IEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,F0HM003BUKIEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,90VT008HUKIEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,21D8006SUKIEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,21HFCTO1WWIE3IEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,21K5CTO1WWIE2IEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,30H0CTO1WWIE2IEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,83BGCTO1WWGB1GBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,83E0003KUKGBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,83E0002TUKGBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,83E2000KUKGBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,83DHCTO1WWGB2GBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,83DFCTO1WWGB2GBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,83DECTO1WWGB3GBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,21L50014UKGBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,21HM0077UKGBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,21HQCTO1WWGB3GBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,21JNCTO1WWGB2GBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,21KGCTO1WWGB2GBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,21KHCTO1WWGB3GBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,21J8CTO1WWGB2GBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,21D8006SUKGBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,83E2000UUKGBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,83E3001YUKGBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,83CV0030UKGBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,83AC000FUKGBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,21HFCTO1WWGB3GBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,21K5CTO1WWGB2GBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,30H0CTO1WWGB2GBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,21F8CTO1WWGB2GBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,21HMCTO1WWGB2GBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,21KC005RUKGBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,21MS0047UKGBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,21M7002MUKGBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,21MA001VUKGBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,21FACTO1WWGB2GBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,M10PLUS3UK2GBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,ZAAJ0385GBGBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,ZAD00071GBGBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,ZACH0200GBGBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,83DE0002UKGBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,82Y5CTO1WWGB2GBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,83DG0046UKGBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,83DGCTO1WWGB1GBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,90UY002VUKGBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,83DA001UUKGBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,83DA001VUKGBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,12E3CTO1WWGB2GBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,12EHCTO1WWGB2GBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,12HKCTO1WWGB2GBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,12E9CTO1WWGB2GBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,12DTCTO1WWGB2GBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,F0HM003BUKGBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,90VT008HUKGBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,LEN101T0066GBEN_GB86676B8CB1BF5-4603-B09F-CC1A51E67D7A,LEN101T0071IEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,83EDCTO1WWIE1IEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,83EDCTO1WWIE2IEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,83EDCTO1WWIE3IEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5,83EDCTO1WWIE4IEEN_IE4157381FBA170-41AC-9AF7-6D6C2E6D29D5

4. However, these rules can still be accessed via the rules API endpoint, allowing one to view, delete, or modify the rules as needed. This will not impact search functionality, but the rules will not be loaded in the Fusion UI.
5. In order to make the rules visible in the UI and make them accessible, we will need to push back all of the offending rules to the second page using dummy rules, then the targeted-docs request will no longer be too long to process and the UI will once again load without issue.
   Example curl to load dummy rules, this is mostly done through curl/postman
   

   curl --location 'http://34.83.233.21:6764/api/apps/TestCollection/query-rewrite/instances' \
   --header 'Content-Type: application/json' \
   --header 'Authorization: Basic <PASSWORD>' \
   --data '{"name": "dummy00",
   "type": "banner",
   "banner_url": "www.google.com",
   "matching":"keywords",
   "search_terms":["TTHISSRRULESSHOULDNNEEVERFFIRE"]
   }

6. Please note that once the rules have been POST’d to the /instances endpoint, the rules UI will be loaded successfully and all the unpublished rules will only be visible to the user who defined it.
7. So the permanent fix for this issue is to actually to replace the targeted-docs call from GET to POST and this has been fixed as part of fusion 5.9.3 as per release notes doc.

Reference doc: 

1. https://doc.lucidworks.com/fusion/5.9/lqgbgi/fusion-5-9-3