Population slow on 150M database

Description

After further optimisation to the population query, the population appears still to be slow on the 150M nodes cloud database. The strange thing is that if the population query is executed directly on the database via a query everything is pretty fast.
If the population is executing via the code, on a 5K nodes slice, it takes at the moment an average of 5 minutes. The same population, executed with a bare SQL statement on the database CLI, takes an average of .2 seconds. this task is to capture the investigation (and the solution) to such issue.

Activity

Show:
Bruno Bossola
February 3, 2021, 5:37 PM

Bruno Bossola
February 2, 2021, 10:32 AM

Started ~11:36, took ~3 hours. We can probably get better performances moving the slice to 10K or 20K.

Bruno Bossola
February 2, 2021, 10:32 AM

The population of a 5K slice takes now 300ms, as expected

Bruno Bossola
February 1, 2021, 10:42 AM
Edited

The query is now logged at DEBUG level for further analysis:

Bruno Bossola
February 1, 2021, 10:40 AM

In the last test on the 150M we:

  • added a JDBC direct execution

  • separated the context creation from the slice population

Those two changes did not make any difference from the query. The only remaining issue is the left+inner for the owner selection, this is the next test to be executed.

Done

Assignee

Unassigned

Reporter

Bruno Bossola

Labels

None

Release Train

Iver

Delivery Team

Team 5

Sprint

None

Fix versions

Priority

Unprioritized