OPIK-859: Reduce find spans query cost

[thiagohora]

Details

Despite the slight differences in the parts and the big difference in granules. I'm confident this new query is more efficient since it postpones the retrieval of fields that are not part of the sortable key. This has sped up the process considerably.

Before:

Expression (Project names)
  Limit
    LimitBy
      Expression ((Before LIMIT BY + (Before ORDER BY + Projection) [lifted up part]))
        Sorting (Sorting for ORDER BY)
          Expression ((Before ORDER BY + Projection))
            Expression
              ReadFromMergeTree (opik_prod.spans)
              Indexes:
                PrimaryKey
                  Keys: 
                    workspace_id
                    project_id
                  Condition: and((workspace_id in ['48d64607-f70d-4cb1-9207-dd658a423f8e', '48d64607-f70d-4cb1-9207-dd658a423f8e']), (project_id in ['019484c0-1a3c-7987-b2d4-0e3598e8717a', '019484c0-1a3c-7987-b2d4-0e3598e8717a']))
                  Parts: 16/20
                  Granules: 828/29114

After:

CreatingSets (Create sets before main query execution)
  Expression ((Project names + (Before ORDER BY + Projection) [lifted up part]))
    Sorting (Sorting for ORDER BY)
      Expression ((Before ORDER BY + Projection))
        Expression
          ReadFromMergeTree (opik_prod.spans)
          Indexes:
            PrimaryKey
              Keys: 
                id
              Condition: (id in 200-element set)
              Parts: 14/14
              Granules: 25196/29116

Issues

OPIK-859

[andrescrz]

This basically converts the JOIN to a SUBQUERY which seems to be more efficient in ClickHouse, or at least for this particular case.

Left some comments of things to double check or for the future, but we should be good to go.

[andrescrz]

Do you still need this duration_millis field here?

[thiagohora]

Because there is a dynamic filter that assumes this field exists in the query. We can probably remove it by changing it to a materialized column instead. This will also make the query simpler

[andrescrz]

Ok, I was no clear about its use case. I know now. No need to change anything at the moment.

[andrescrz]

In the future, we can filter by workspace_id here as well.

[thiagohora]

Agree I will push this in a following PR

[andrescrz]

Let's double check if we need a similar optimisation for the related count query.

[thiagohora]

The count follows the same structure as we are using now (subquery returning id and duration_millis), and then count only one ìd. So we should be fine.