-
Notifications
You must be signed in to change notification settings - Fork 14.6k
New issue
Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? # to your account
Adding run_id column to log table #37731
Conversation
6656f1e
to
b04aa58
Compare
b04aa58
to
1052412
Compare
I think prepopulating the runId filter works. After this I will open up the before/after fields to be user-editable but with some default options like a dag run's entire durations. The before/after range doesn't capture changing the state or a note. so either approach has pros and cons Also, after this and #37734. We probably want to make sure we are populating run_id everywhere we can. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM . But would love another maintainer's review
Just a watch-out - we have two migrations in-progress. I think we have a protection that will make the secon one not mergable if the first is but there will be some re-work needed once the other mighration is merged #37708 |
We actually have 3 in flight 😅: #37734 |
😱 😱 😱 😱 😱 😱 😱 😱 😱 |
Thanks for the heads up - I'll keep a close 👀 on the other two and try to keep this PR in a mergeable state. |
Since we have two migrations touching Log. Should we do both at once? |
This is a pretty good idea. Which PR do you want to merge first? If needed I can port the DB changes from the other PR into this one. |
The other Audit Log one still has errors in the CI. So let's merge this first. The only change I had to make over there is to increase the max size of the |
Looks like the other TaskInstance migration PR was merged, I'll get this one updated. |
9f2400a
to
45e11e4
Compare
45e11e4
to
ebbc05f
Compare
@jedcunningham @potiuk What do you prefer? Two separate event log table migrations, or combine them? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm. We'll merge the two migrations in the second PR
closes: #37597
This adds the
run_id
column to the log table (without a backfill, see discussion) and exposes it in all of the different surfaces:/eventLogs
API/logs/list
)Screenshots:
Dag-scoped audit logs:
![image](https://private-user-images.githubusercontent.com/16950874/307991496-13a451b4-7b61-43b9-94a2-c6131a17e157.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkyODQ0NDksIm5iZiI6MTczOTI4NDE0OSwicGF0aCI6Ii8xNjk1MDg3NC8zMDc5OTE0OTYtMTNhNDUxYjQtN2I2MS00M2I5LTk0YTItYzYxMzFhMTdlMTU3LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAyMTElMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMjExVDE0MjkwOVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPThmZDRlMDI3NGExMjJhODVmMDQxMjBiYzg3ZjMyZjRjYjM3Njk3ZTNjZjg3NDcwOGU0OTdmMzExNzQyZWJlYjQmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.2Z4_yYIOHtvW5cJymhaTKHq9q4cZc2qZfgH459paPsc)
Dagrun-scoped audit logs:
![image](https://private-user-images.githubusercontent.com/16950874/307991407-b1853d89-d819-4228-a5d7-2f1c7f1207a8.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkyODQ0NDksIm5iZiI6MTczOTI4NDE0OSwicGF0aCI6Ii8xNjk1MDg3NC8zMDc5OTE0MDctYjE4NTNkODktZDgxOS00MjI4LWE1ZDctMmYxYzdmMTIwN2E4LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAyMTElMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMjExVDE0MjkwOVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTIwMzRlN2MzYzNlODE1MTE2ZTIyZTZiNjBmYzU4NTA1NjJmOTU5ZDYzYmUwZTA2NGRhM2EwZTBmY2UwZDE1NDgmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.8DgaQ25xotfmqGcmMKx-OaXnpbRafbgVjmpvJq80S3k)
Open questions: