Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

[jaeger-v2] Refactor ElasticSearch/OpenSearch Storage Configurations #6060

Merged
merged 45 commits into from
Oct 12, 2024
Merged

[jaeger-v2] Refactor ElasticSearch/OpenSearch Storage Configurations #6060

merged 45 commits into from
Oct 12, 2024

Conversation

mahadzaryab1
Copy link
Collaborator

@mahadzaryab1 mahadzaryab1 commented Oct 6, 2024 •
edited
Loading

Which problem is this PR solving?

Description of the changes

How was this change tested?

  • Unit Tests and E2E Integration Tests

Checklist

@codecov Codecov
Copy link

codecov bot commented Oct 6, 2024 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.91%. Comparing base (b2f8be2) to head (bfd0f93).
Report is 1 commits behind head on main.

Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #6060      +/-   ##
==========================================
- Coverage   96.92%   96.91%   -0.01%     
==========================================
  Files         349      349              
  Lines       16598    16646      +48     
==========================================
+ Hits        16087    16133      +46     
- Misses        328      329       +1     
- Partials      183      184       +1
Flag Coverage Δ
badger_v1 7.98% <0.00%> (-0.01%) ⬇️
badger_v2 1.84% <0.00%> (+0.02%) ⬆️
cassandra-4.x-v1 15.75% <0.00%> (-0.04%) ⬇️
cassandra-4.x-v2 1.77% <0.00%> (+0.02%) ⬆️
cassandra-5.x-v1 15.75% <0.00%> (-0.04%) ⬇️
cassandra-5.x-v2 1.77% <0.00%> (+0.02%) ⬆️
elasticsearch-6.x-v1 18.95% <86.24%> (+0.25%) ⬆️
elasticsearch-7.x-v1 19.01% <86.24%> (+0.26%) ⬆️
elasticsearch-8.x-v1 19.20% <86.24%> (+0.25%) ⬆️
elasticsearch-8.x-v2 1.83% <0.00%> (+0.01%) ⬆️
grpc_v1 9.35% <0.00%> (-0.03%) ⬇️
grpc_v2 7.12% <0.00%> (+<0.01%) ⬆️
kafka-v1 9.68% <0.00%> (-0.02%) ⬇️
kafka-v2 1.84% <0.00%> (+0.02%) ⬆️
memory_v2 1.84% <0.00%> (+0.02%) ⬆️
opensearch-1.x-v1 19.06% <86.24%> (+0.26%) ⬆️
opensearch-2.x-v1 19.05% <86.24%> (+0.24%) ⬆️
opensearch-2.x-v2 1.83% <0.00%> (+0.01%) ⬆️
tailsampling-processor 0.49% <0.00%> (+0.03%) ⬆️
unittests 95.72% <100.00%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

JaredTan95 and others added 10 commits October 6, 2024 10:48
@JaredTan95 @mahadzaryab1
Add configurable index data layout and rollover frequency
51bdde1 
Signed-off-by: Jared Tan <jian.tan@daocloud.io>
Signed-off-by: Mahad Zaryab <mahadzaryab1@gmail.com>
@JaredTan95 @mahadzaryab1
remove indexprefix
b3b0559 
Signed-off-by: Jared Tan <jian.tan@daocloud.io>
Signed-off-by: Mahad Zaryab <mahadzaryab1@gmail.com>
@JaredTan95 @mahadzaryab1
fix prefix
6dbb41e 
Signed-off-by: Jared Tan <jian.tan@daocloud.io>
Signed-off-by: Mahad Zaryab <mahadzaryab1@gmail.com>
@JaredTan95 @mahadzaryab1
fix prefix
44ea7cb 
Signed-off-by: Jared Tan <jian.tan@daocloud.io>
Signed-off-by: Mahad Zaryab <mahadzaryab1@gmail.com>
@JaredTan95 @mahadzaryab1
fix prefix
c0a3040 
Signed-off-by: Jared Tan <jian.tan@daocloud.io>
Signed-off-by: Mahad Zaryab <mahadzaryab1@gmail.com>
@yurishkuro @mahadzaryab1
fix-opensearch-config
4498844 
Signed-off-by: Yuri Shkuro <github@ysh.us>
Signed-off-by: Mahad Zaryab <mahadzaryab1@gmail.com>
@mahadzaryab1
Add Function To Get Index Name--signoff
88928fb 
Signed-off-by: Mahad Zaryab <mahadzaryab1@gmail.com>
@mahadzaryab1
Use New Function
60cc5a3 
Signed-off-by: Mahad Zaryab <mahadzaryab1@gmail.com>
@mahadzaryab1
Remove Redundant Constants
aec39e5 
Signed-off-by: Mahad Zaryab <mahadzaryab1@gmail.com>
@mahadzaryab1
Fix Test
4663d80 
Signed-off-by: Mahad Zaryab <mahadzaryab1@gmail.com>
@mahadzaryab1
Remove Embedding
eb14f4f 
Signed-off-by: Mahad Zaryab <mahadzaryab1@gmail.com>
@mahadzaryab1
Fix Missing Template Options
6200e26 
Signed-off-by: Mahad Zaryab <mahadzaryab1@gmail.com>
@mahadzaryab1
Fix Missing Template Options
9802dff 
Signed-off-by: Mahad Zaryab <mahadzaryab1@gmail.com>
@mahadzaryab1
Fix Naming In Configuration File
5976d42 
Signed-off-by: Mahad Zaryab <mahadzaryab1@gmail.com>
mahadzaryab1
mahadzaryab1 commented Oct 6, 2024
View reviewed changes
mahadzaryab1
mahadzaryab1 commented Oct 6, 2024
View reviewed changes
@mahadzaryab1 mahadzaryab1 changed the title [WIP] Refactor ElasticSearch/OpenSearch Storage Configurations Refactor ElasticSearch/OpenSearch Storage Configurations Oct 6, 2024
@mahadzaryab1 mahadzaryab1 marked this pull request as ready for review October 6, 2024 15:44
@mahadzaryab1 mahadzaryab1 requested a review from a team as a code owner October 6, 2024 15:44
@mahadzaryab1 mahadzaryab1 mentioned this pull request Oct 6, 2024
Closed
16 tasks
@yurishkuro yurishkuro mentioned this pull request Oct 6, 2024
Closed
@yurishkuro yurishkuro added the changelog:exprimental Change to an experimental part of the code label Oct 6, 2024
@mahadzaryab1
Add TODO
b4a0456 
Signed-off-by: Mahad Zaryab <mahadzaryab1@gmail.com>
@mahadzaryab1
Copy link
Collaborator Author

@yurishkuro regarding #5790 (comment), are you referring to https://github.com/jaegertracing/jaeger/blob/main/plugin/storage/es/factory.go#L196-L210?

@yurishkuro
Copy link
Member

regarding #5790 (comment), are you referring to https://github.com/jaegertracing/jaeger/blob/main/plugin/storage/es/factory.go#L196-L210?

yes

mahadzaryab1 and others added 3 commits October 8, 2024 07:51
@mahadzaryab1
Fix Comparison
b2a15b4 
Signed-off-by: Mahad Zaryab <mahadzaryab1@gmail.com>
@mahadzaryab1
Fix Unit Tests And Change Types of IndexPrefix
86b0c3a 
Signed-off-by: Mahad Zaryab <mahadzaryab1@gmail.com>
@mahadzaryab1
Merge branch 'main' into es-layout
c95a173
@mahadzaryab1
Address Feedback From PR Review
1b4d8c0 
Signed-off-by: Mahad Zaryab <mahadzaryab1@gmail.com>
@mahadzaryab1
Address Feedback From PR Review
c83f841 
Signed-off-by: Mahad Zaryab <mahadzaryab1@gmail.com>
mahadzaryab1 and others added 2 commits October 10, 2024 19:58
@mahadzaryab1
Add Comment To Config Struct
d862b6f 
Signed-off-by: Mahad Zaryab <mahadzaryab1@gmail.com>
@mahadzaryab1
Merge branch 'main' into es-layout
bfd0f93
@yurishkuro yurishkuro merged commit 563f3b0 into jaegertracing:main Oct 12, 2024
49 checks passed
Saumya40-codes pushed a commit to Saumya40-codes/jaeger that referenced this pull request Oct 14, 2024
@mahadzaryab1 @JaredTan95 @yurishkuro
[jaeger-v2] Refactor ElasticSearch/OpenSearch Storage Configurations (j…
dff5ed5 
…aegertracing#6060)

## Which problem is this PR solving?
- Part of jaegertracing#6059

## Description of the changes
- Continuation of jaegertracing#5790
- Refactors the configurations for ElasticSearch and OpenSearch to
simplify the configuration by having more logical groupings

## How was this change tested?
- Unit Tests and E2E Integration Tests

## Checklist
- [x] I have read
https://github.com/jaegertracing/jaeger/blob/master/CONTRIBUTING_GUIDELINES.md
- [x] I have signed all commits
- [x] I have added unit tests for the new functionality
- [x] I have run lint and test steps successfully
  - for `jaeger`: `make lint test`
  - for `jaeger-ui`: `yarn lint` and `yarn test`

---------

Signed-off-by: Jared Tan <jian.tan@daocloud.io>
Signed-off-by: Mahad Zaryab <mahadzaryab1@gmail.com>
Signed-off-by: Yuri Shkuro <github@ysh.us>
Co-authored-by: Jared Tan <jian.tan@daocloud.io>
Co-authored-by: Yuri Shkuro <github@ysh.us>
Signed-off-by: Saumya Shah <saumyabshah90@gmail.com>
Saumya40-codes pushed a commit to Saumya40-codes/jaeger that referenced this pull request Oct 14, 2024
@mahadzaryab1 @JaredTan95
@yurishkuro @Saumya40-codes
[jaeger-v2] Refactor ElasticSearch/OpenSearch Storage Configurations (j…
2b0237e 
…aegertracing#6060)

## Which problem is this PR solving?
- Part of jaegertracing#6059

## Description of the changes
- Continuation of jaegertracing#5790
- Refactors the configurations for ElasticSearch and OpenSearch to
simplify the configuration by having more logical groupings

## How was this change tested?
- Unit Tests and E2E Integration Tests

## Checklist
- [x] I have read
https://github.com/jaegertracing/jaeger/blob/master/CONTRIBUTING_GUIDELINES.md
- [x] I have signed all commits
- [x] I have added unit tests for the new functionality
- [x] I have run lint and test steps successfully
  - for `jaeger`: `make lint test`
  - for `jaeger-ui`: `yarn lint` and `yarn test`

---------

Signed-off-by: Jared Tan <jian.tan@daocloud.io>
Signed-off-by: Mahad Zaryab <mahadzaryab1@gmail.com>
Signed-off-by: Yuri Shkuro <github@ysh.us>
Co-authored-by: Jared Tan <jian.tan@daocloud.io>
Co-authored-by: Yuri Shkuro <github@ysh.us>
Signed-off-by: Saumya Shah <saumyabshah90@gmail.com>
chahatsagarmain pushed a commit to chahatsagarmain/jaeger that referenced this pull request Oct 23, 2024
@mahadzaryab1 @JaredTan95
@yurishkuro @chahatsagarmain
[jaeger-v2] Refactor ElasticSearch/OpenSearch Storage Configurations (j…
293819e 
…aegertracing#6060)

## Which problem is this PR solving?
- Part of jaegertracing#6059

## Description of the changes
- Continuation of jaegertracing#5790
- Refactors the configurations for ElasticSearch and OpenSearch to
simplify the configuration by having more logical groupings

## How was this change tested?
- Unit Tests and E2E Integration Tests

## Checklist
- [x] I have read
https://github.com/jaegertracing/jaeger/blob/master/CONTRIBUTING_GUIDELINES.md
- [x] I have signed all commits
- [x] I have added unit tests for the new functionality
- [x] I have run lint and test steps successfully
  - for `jaeger`: `make lint test`
  - for `jaeger-ui`: `yarn lint` and `yarn test`

---------

Signed-off-by: Jared Tan <jian.tan@daocloud.io>
Signed-off-by: Mahad Zaryab <mahadzaryab1@gmail.com>
Signed-off-by: Yuri Shkuro <github@ysh.us>
Co-authored-by: Jared Tan <jian.tan@daocloud.io>
Co-authored-by: Yuri Shkuro <github@ysh.us>
@mahadzaryab1 mahadzaryab1 deleted the es-layout branch October 31, 2024 22:35
@yurishkuro yurishkuro mentioned this pull request Nov 27, 2024
Closed
# for free to join this conversation on GitHub. Already have an account? # to comment
Reviewers

@yurishkuro yurishkuro yurishkuro approved these changes

@albertteoh albertteoh Awaiting requested review from albertteoh albertteoh is a code owner automatically assigned from jaegertracing/jaeger-maintainers

Assignees
No one assigned
Labels
changelog:exprimental Change to an experimental part of the code storage/elasticsearch
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@mahadzaryab1 @yurishkuro @JaredTan95