Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Issue #378: Add alembic support for database migrations #387

Draft
wants to merge 98 commits into
base: master
Choose a base branch
from
Draft

Issue #378: Add alembic support for database migrations #387

wants to merge 98 commits into from

Conversation

baileythegreen
Copy link
Contributor

@baileythegreen baileythegreen commented Mar 28, 2022 •
edited
Loading

This PR adds alembic for pyani, making it possible to upgrade and downgrade existing databases to accommodate otherwise breaking changes to the ORM, as with the addition of fastANI capabilities.

Closes #378.

Type of change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality not to work as expected)
  • This change requires a documentation update
  • This is a documentation update

Action Checklist

  • Work on a single issue/concept (if there are multiple separate issues to address, please use a separate pull request for each)
  • Fork the pyani repository under your own account (please allow write access for repository maintainers)
  • Set up an appropriate development environment (please see CONTRIBUTING.md)
  • Create a new branch with a short, descriptive name
  • Work on this branch
    • style guidelines have been followed
    • new code is commented, especially in hard-to-understand areas
    • corresponding changes to documentation have been made
    • tests for the change have been added that demonstrate the fix or feature works
  • Test locally with pytest -v non-passing code will not be merged
  • Rebase against origin/master
  • Check changes with flake8 and black before submission
  • Commit branch
  • Submit pull request, describing the content and intent of the pull request
  • Request a code review
  • Continue the discussion at Pull requests section in the pyani repository

@baileythegreen
Basic infrastructure for versiondb subcommand
2934ad5
@baileythegreen
Add new requirement
cbc6a71
@baileythegreen
Add get_version() for alembic
8fcd224
@baileythegreen
Hook pyani versiondb infrastructure to functional code
e7a5c30
@baileythegreen
Add generic versions of files required for alembic to function
6b88b2b
@baileythegreen
Alter handling of stderr and stdoutfrom alembic
bfd6a60
@baileythegreen
Add logging and use environmental variables
ea6adf1
@baileythegreen
Add migration script for fastANI changes to the ORM
aed669d
@baileythegreen
Backup existing database before migrating
3d66953
@baileythegreen
Use a local file to track versions in dry-run mode
4a35636
@baileythegreen
Update parser to use const values instead of defaults
18788ed
@baileythegreen
Update versiondb to accommodate --dry-run option
0531a65
@baileythegreen
Clean up subcmd_versiondb
b297630
@baileythegreen
Add initial tests for versiondb
51bac6f
@baileythegreen
Fix test comparison value
b37a1ca
@baileythegreen
Specify which parser arguments are optional
b2c74f4
@baileythegreen
Abstract migration functions to one and implement dry-run
0b9d47a
@baileythegreen
Add alembic_version table to database model
7f933ba
@baileythegreen
Add current database version to alembic_version table
216778a
@baileythegreen
Update .gitignore to ignore some testing files
ac061d5
@baileythegreen
Change unique constraints for comparisons table with Alembic
f875508
@baileythegreen
Groundwork for tests/test_versiondb.py
f8140b9
@baileythegreen
Add timestamp to dry-run output file name
2c3500c
@baileythegreen
Update migration file to allow for reflection in offline mode
9c642b3
@baileythegreen
Add empty database fixtures for tests
79e06f2
@baileythegreen
Update test_versiondb.py input file names
54e814f
@baileythegreen
Add docs for versiondb
a747d46
@baileythegreen
Add docs for versiondbAdd a directory for versionb` output files
4ea0b67
@baileythegreen
Rework tests to look for meaningful differences between output and ta…
51fb696 
…rgets
@baileythegreen
Update starting point for database downgrade tests
871a149
@codecov
Copy link

codecov bot commented Mar 28, 2022 •
edited
Loading

Codecov Report

Merging #387 (582bdc4) into master (62949c4) will decrease coverage by 0.53%.
The diff coverage is 63.95%.

@@            Coverage Diff             @@
##           master     #387      +/-   ##
==========================================
- Coverage   75.82%   75.28%   -0.54%     
==========================================
  Files          55       58       +3     
  Lines        3747     3917     +170     
==========================================
+ Hits         2841     2949     +108     
- Misses        906      968      +62

@baileythegreen
Copy link
Contributor Author

This PR is still missing tests for the --dry-run option and some investigation is needed to ensure it works as expected in a non-development environment (where pyani versiondb would not be called from a dev pyani directory, and so the requisite alembic config files would be located elsewhere.

@baileythegreen
Copy link
Contributor Author

In the interest of having the database migration functionality merged sooner, I would suggest that tests for the --dry-run option here be made a separate issue. The --dry-run option is neither the main purpose of this issue, nor does the migration capability hinge upon the --dry-run working correctly.

@baileythegreen baileythegreen added this to the 0.3.0 milestone May 4, 2022
@baileythegreen baileythegreen added the enhancement something we'd like pyani to do that it doesn't already label May 11, 2022
# for free to join this conversation on GitHub. Already have an account? # to comment
Reviewers

@widdowquinn widdowquinn Awaiting requested review from widdowquinn widdowquinn will be requested when the pull request is marked ready for review widdowquinn is a code owner

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
enhancement something we'd like pyani to do that it doesn't already
Projects
None yet
Milestone
0.3.0
Development

Successfully merging this pull request may close these issues.

Use alembic to manage SQLite schema changes?
1 participant
@baileythegreen