Ask the Community
Groups
Stale Results - Connect IT Community | Kaseya
<main> <article class="userContent"> <p><strong>PROBLEM:</strong></p> <p>In STATUS >> DEVICES bottom right pane, if your test results have a triangle with an exclamation point inside of it displayed next to them, then this indicates that the results are old/stale and have not been written to the database recently. </p> <p>Technically speaking, a stale result indicates that the test poll value has not been written to the database in at least 3 poll cycles. e.g. A test with a 5 minute poll interval that has not written a test result in at least 16 minutes will display as stale.</p> <p><strong>SOLUTION:</strong></p> <p>The most likely causes of stale results are usually caused by:</p> <p>a) BVE/DGE/DGEx's server times are not synchronized.</p> <p>b) A table in the MySQL database, on the monitoring/upstream DGE, needs repairing or the database needs to be optimized.</p> <p>c) DGEx is unable to connect to the upstream DGE.</p> <p><img src="https://us.v-cdn.net/6032361/uploads/migrated/OEW29U243W0Q/stale-results.png" alt="Stale_Results.png" class="embedImage-img importedEmbed-img"></img></p> <p> </p> <p><strong>Please check the following:</strong></p> <p><strong>(A) BVE/DGE/DGEx's server times are not synchronized</strong></p> <p>If the time-offsets are significant, results may not be processed as expected or in a timely manner.</p> <p>* Check SUPERUSER >> HEALTH</p> <p>* If you see any Time offsets in SUPERUSER >> HEALTH, please correct as per <a rel="nofollow" href="https://kaseya.vanillacommunities.com/kb/articles/aliases/kaseya/hc/en-gb/articles/115005795267-Traverse-Component-Health-page-shows-non-zero-time-offsets">Traverse Component Health Page Shows Non-Zero Time Offsets</a></p> <p> </p> <p><strong>(B) A table in the MySQL database, on the monitoring/upstream DGE, needs repairing or the database needs to be optimized</strong></p> <p>* Check the test "Writer Queue size" for the DGE monitoring the device with 'Stale Results'. The 'Writer Queue Size (AggregatedDataDbWriter0)' test is expected to be historically high compared to normal, which could point to a database repair/optimize being required.</p> <p>* Check <TraverseHome>\logs\mysql.log and see if it has entries containing:</p> <pre class="code codeBlock" spellcheck="false" tabindex="0">Table 'XXXXX' is marked as crashed and last (automatic?) repair failed </pre> <p><strong>If it does, it will require a database repair, followed by an optimize.</strong></p> <p>* Check if either the BVE or DGE that is monitoring these 'Stale Results' requires a repair: </p> <pre class="code codeBlock" spellcheck="false" tabindex="0">cd C:\Program Files (x86)\Traverse\utils<br>db_optimize.pl --info</pre> <p>Scroll to the very top and if you see the below, the database requires a repair:</p> <pre class="code codeBlock" spellcheck="false" tabindex="0">"Table 'XXXXX' is marked as crashed and last (automatic?) repair failed"</pre> <p>If the above output is not visible (due to too many lines outputted). Then you can pipe the output to a file</p> <pre class="code codeBlock" spellcheck="false" tabindex="0">db_optimize.pl --info >> output.txt</pre> <p><strong>If a repair is required, please follow <a rel="nofollow" href="https://kaseya.vanillacommunities.com/kb/articles/aliases/kaseya/hc/en-gb/articles/229041468-How-Do-I-Repair-An-Error-with-a-Traverse-Database">How Do I Repair An Error With A Traverse Database</a> followed by an Optimize <a rel="nofollow" href="https://kaseya.vanillacommunities.com/kb/articles/aliases/kaseya/hc/en-gb/articles/229045848">Tip: How Do I optimize the DGE Database</a>. </strong></p> <p>Bear in mind that this may take several hours to complete. It is advised to tail the database_repair.log in a separate command prompt so you can confirm it's running. The full list of Windows commands are repeated below for your convenience.</p> <p> </p> <pre class="code codeBlock" spellcheck="false" tabindex="0"><strong>On the affected DGE:</strong><br>1) Stop all Traverse services<br>2) Run the database repair<br>cd <TraverseHome>\utils<br>db_repair.cmd</pre> <pre class="code codeBlock" spellcheck="false" tabindex="0"><strong>Tail the database_repair.log in a separate window to see the progress of the repair:</strong><br>cd <TraverseHome>\logs<br>tail -f database_repair.log</pre> <p>* <strong>If a repair was carried out, then an optimize will be required afterwards.</strong> Even if a repair was not required, running a db_optimize would be advisable, especially if "Writer Queue size" test on the DGE is high, as this is an indicator of a non-optimized database. It will complete immediately if it was not required. <a rel="nofollow" href="https://kaseya.vanillacommunities.com/kb/articles/aliases/kaseya/hc/en-gb/articles/229045848-Tip-How-Do-I-optimize-the-DGE-Database">Tip: How Do I Optimize The DGE Database</a>. This can also possibly take several hours.</p> <pre class="code codeBlock" spellcheck="false" tabindex="0">3) Once the repair finishes (the cursor returns)<br>Start all Services<br>4) Run a database optimization<br>cd <TraverseHome>\utils<br>db_optimize.pl --run</pre> <p>Once the repair/optimize has completed, it will take time for the backlog of results to be written to the database. Take note that DGEx's can store up to 12 hours of results and will write the oldest results to the database first. It can take some time for these test results to 'catch up'.</p> <p><strong>To confirm that the results are now being written to the database:</strong></p> <pre class="code codeBlock" spellcheck="false" tabindex="0">1) Navigate to STATUS >> DEVICES<br>2) Click on the 'Blue URL' device name</pre> <p><img src="https://us.v-cdn.net/6032361/uploads/migrated/9B1IKQBXT852/mceclip0.png" width="987" height="358" alt="image" class="embedImage-img importedEmbed-img"></img></p> <pre class="code codeBlock" spellcheck="false" tabindex="0">3) This page will display the timestamp of the most recently written test result for <br>each individual test. Hitting refresh occasionally, you should start to see the <br>timestamps of the tests advance to within a polling cycle of the current time.</pre> <p><img src="https://us.v-cdn.net/6032361/uploads/migrated/YQEJQAYDDY5T/mceclip2.png" width="999" height="523" alt="image" class="embedImage-img importedEmbed-img"></img></p> <p> </p> <p> </p> <p><strong>(C) DGEx is unable to connect to the upstream DGE</strong></p> <p>Please log onto the monitoring DGEx and check the current date error log located at</p> <p>* <TraverseHome>\logs\monitor\error-20yy-mm-dd.log</p> <p>and look for any entries of the below:</p> <p>* (WARN) Unable to communicate with upstream DGE; Results will be stored locally while attempting to restore connection</p> <p>If the DGEx has lost connectivity, please ensure that all the required ports are open on the firewall. <strong>Port 9443 is responsible for forwarding test results to the upstream DGE so please confirm that it is open</strong> as per the below articles:</p> <p><a rel="nofollow" href="https://kaseya.vanillacommunities.com/kb/articles/aliases/kaseya/hc/en-gb/articles/229044868-Communication-between-Traverse-servers">Communication between Traverse servers</a></p> <p><a rel="nofollow" href="/home/leaving?allowTrusted=1&target=http%3A%2F%2Fhelp.kaseya.com%2FWebHelp%2FEN%2Ftv%2F9050000%2Findex.asp%2316379.htm">Using Traverse Behind Firewalls</a></p> <p><a rel="nofollow" href="https://kaseya.vanillacommunities.com/kb/articles/aliases/kaseya/hc/en-gb/articles/229042148">Tests not updating</a></p> <p>If the DGEx is constantly dropping and reconnecting to the upstream DGE, this would usually indicate a networking issue between the DGE and the DGEx. It is advisable to investigate and resolve this networking issue to ensure Traverse stability.</p> <p>Once you have corrected the DGEx to DGE connectivity issue, then as with (b), please review STATUS >> DEVICES to confirm that results are catching up.</p> </article> </main>