[candidate] Create Sex table and remove Sex enum #9025

CamilleBeau · 2024-01-29T20:14:34Z

Brief summary of changes

This PR changed the datatype of the candidate table columns "Sex" and ProbandSex" from enum to varchar(255), instead creating a constraint on the columns determined by a new "sex" table.

The "sex" table contains optional values for candidate sex and candidate proband sex. Projects can add or remove Sex options by simply inserting into or dropping from this table.

I also modified anywhere that I could find that was hardcoding Sex options to instead get the sex options from the database through a Utility class function.

Have you updated related documentation?

Testing instructions (if applicable)

Test creating a new candidate. Make sure that the sex is inserted into the candidate table properly.
Set "use proband" config setting to true in the config module, and then test the proband tab of candidate parameters
Go to the candidate list module and select "Show Advanced Filters". Make sure that the Sex filter contains the right values and works properly.
Check the cnv, methylation, profiles, and snp tabs all filter for Sex properly, and that the sex dropdown contains the correct values.
Add or drop a value from the Sex table, and repeat steps 1 through 5 making sure that the drop-downs are populated with the updated sex options.
Attempt update a candidate to have a Sex and ProbandSex that do not exist in the sex table. Make sure you get an error.
Attempt to delete a value from the sex table that is used in the candidate table. Make sure you get an error.
Test the CouchDB_Import_Demographics.php script

Link(s) to related issue(s)

Resolves Sex should not be an enum #8936

CamilleBeau · 2024-01-29T20:55:52Z

@kongtiaowang I may need a bit of help with the test suites here. It looks like I'm going to have to add the Sex table to the mock data in these tests, but I don't understand how to do this very well

kongtiaowang · 2024-01-30T15:42:00Z

@CamilleBeau I will debug the test issue later, after #9026 which it will fix the api test issue.

CamilleBeau · 2024-01-30T15:53:19Z

@kongtiaowang Thanks so much!

maximemulder · 2024-02-02T14:24:20Z

SQL/0000-00-00-schema.sql

@@ -72,6 +72,15 @@ CREATE TABLE `language` (
 INSERT INTO language (language_code, language_label) VALUES
    ('en-CA', 'English');

+CREATE TABLE `sex` (
+  `SexID` int(10) unsigned NOT NULL AUTO_INCREMENT,
+  `SexValue` varchar(255) NOT NULL,


Nitpicky but as I said during the meeting, I find "SexValue" to be a little generic, and think something like "SexName" would be more descriptive (this also applies to the rest of this PR obviously).

Updated "SexValue" to "Sex", and also modified charset to utf8mb4

CamilleBeau · 2024-02-06T15:33:11Z

@kongtiaowang following the fix for api test issue and rebasing, this should be ready for debugging

maximemulder · 2024-02-08T15:12:59Z

SQL/0000-00-00-schema.sql

@@ -72,6 +72,15 @@ CREATE TABLE `language` (
 INSERT INTO language (language_code, language_label) VALUES
    ('en-CA', 'English');

+CREATE TABLE `sex` (
+  `SexID` int(10) unsigned NOT NULL AUTO_INCREMENT,
+  `Sex` varchar(255) NOT NULL,


Hhhhm, my previous comment was just a little thought I had during our meeting, but if I am to really review this pull request I guess I'll try to be more exhaustive.

After giving it more thought, I am still not fully satisfied with the field names. I think having Sex in these names when we are already in the sex table is a little redundant. For instance, I'd really prefer to write sex.ID and sex.Name rather than sex.SexID and sex.Sex in my SQL requests, especially since the first one may look like a foreign key and the second is quite cryptic (imo). So I strongly advocate for the names ID and Name for the field names.

Regarding capitalization, the database sure is messy but I think it would be a good idea to maintain a consistent convention in at least the new tables. I am under the impression that this convention is unofficially singular snake_case for the table names and PascalCase for the field names. I am fine with that and this PR matches this convention, but I am still new and this impression of mine may be wrong, so can someone with more database experience than me confirm or deny it ?

Finally, regarding the pointing of sex foreign keys on ID or Name, I do not have a strong opinion as both have their pros and cons. I think using ID as a foreign key would probably be conceptually cleaner, but also require to update the current database and queries as well as require more joins in the latter. Using Name is certainly easier and will lead to simpler queries.

It would probably have been easier if I had mentionned all that earlier so sorry for the late reply ! In any case, I'd like someone with more database experience than me to also review this pull request.

ERROR 3780 (HY000) at line 155: Referencing column 'Sex' and referenced column 'Sex' in foreign key constraint 'FK_candidate_sex_1' are incompatible.

Regarding capitalization, the database sure is messy but I think it would be a good idea to maintain a consistent convention in at least the new tables. I am under the impression that this convention is unofficially singular snake_case for the table names and PascalCase for the field names. I am fine with that and this PR matches this convention, but I am still new and this impression of mine may be wrong, so can someone with more database experience than me confirm or deny it ?

That seems to match is the official policy documented in docs/SQLModelingStandard.md but it's so weakly enforced that I had to go look it up.

That seems to match is the official policy documented in docs/SQLModelingStandard.md but it's so weakly enforced that I had to go look it up.

Damn I looked at the documentation but somehow missed this file, thanks for pointing it to me. I forgot my glasses at home today so I guess I'm gonna use that as my excuse 😅

driusan · 2024-02-08T16:24:58Z

SQL/0000-00-00-schema.sql

@@ -72,6 +72,15 @@ CREATE TABLE `language` (
 INSERT INTO language (language_code, language_label) VALUES
    ('en-CA', 'English');

+CREATE TABLE `sex` (
+  `SexID` int(10) unsigned NOT NULL AUTO_INCREMENT,


What purpose does this column have? It is not being used as a foreign key by any other tables and the other column in this table already has a not null/unique constraint and is being used as a foreign key into this table.

Removing it would solve the "what should it be named" debate in the thread below.

True, this column is not useful. I don't mind removing it

CamilleBeau · 2024-02-08T16:48:42Z

@driusan @maximemulder I made the updates to change the "Sex" column of sex table to "Name", and removed the "ID" column. It turns out changing the charset of the "sex" table to utf8mb4 caused an issue with the foreign keys because the sex table and candidate table have to have the same charset. I switched it back to utf8.

@kongtiaowang The last test suite errors I believe were caused by the above issue with the charsets differing between tables. That is fixed now, but I expect I will still get test suite errors because of tests in the CandidateTest class which I am still unfamiliar with how to fix.

kongtiaowang · 2024-02-08T17:29:02Z

src/StudyEntities/Candidate/Sex.php

    {
-        return in_array($value, self::VALID_VALUES, true);
+        return in_array($value, $validValues, true);


getSexList will return array(
'Male' => 'Male',
'Female' => 'Female',
'Other' => 'Other'
) so you need to change to [return in_array($value, array_values($validValues), true);]

@kongtiaowang Made that change but still having trouble unfortunately!

modules/candidate_parameters/php/candidatequeryengine.class.inc

test/unittests/CandidateTest.php

CamilleBeau added 3 commits January 30, 2024 16:39

[candidate] Create Sex table and remove Sex enum

2d77b8d

Changelog

36118c2

small wording change

a6fdfaf

CamilleBeau force-pushed the 2024_01_29_sex_remove_enum branch from d6ed2e3 to a6fdfaf Compare January 30, 2024 21:39

maximemulder reviewed Feb 2, 2024

View reviewed changes

Rename SexValue to Sex, and change to mb4

45660ba

CamilleBeau requested a review from maximemulder February 6, 2024 15:31

CamilleBeau assigned maximemulder Feb 6, 2024

CamilleBeau assigned kongtiaowang Feb 6, 2024

kongtiaowang closed this Feb 8, 2024

kongtiaowang reopened this Feb 8, 2024

maximemulder reviewed Feb 8, 2024

View reviewed changes

driusan reviewed Feb 8, 2024

View reviewed changes

Rename values, remove ID column, change charset back

1400e27

kongtiaowang reviewed Feb 8, 2024

View reviewed changes

CamilleBeau and others added 10 commits February 8, 2024 14:25

Change array format

f425084

array_values again

f428603

fix test

644d190

cs

a468632

phan

ee8dc34

test

491362e

phan fix

07bb514

phan

8dc235c

fix

6e39684

last fix

1cb5fcf

driusan reviewed Feb 9, 2024

View reviewed changes

modules/candidate_parameters/php/candidatequeryengine.class.inc Outdated Show resolved Hide resolved

driusan reviewed Feb 9, 2024

View reviewed changes

test/unittests/CandidateTest.php Outdated Show resolved Hide resolved

driusan reviewed Feb 9, 2024

View reviewed changes

test/unittests/CandidateTest.php Outdated Show resolved Hide resolved

kongtiaowang added 9 commits February 12, 2024 10:14

phan

d39be27

fix

cc70c16

try

68e6784

fix phan

b096b34

last try

b8bc55a

format fix

201e100

fix

3a72ae7

fix

547257c

fix

1ae4da3

driusan merged commit e2d6f04 into aces:main Feb 12, 2024
28 checks passed

ridz1208 added this to the 26.0.0 milestone Apr 9, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[candidate] Create Sex table and remove Sex enum #9025

[candidate] Create Sex table and remove Sex enum #9025

CamilleBeau commented Jan 29, 2024 •

edited

Loading

CamilleBeau commented Jan 29, 2024

kongtiaowang commented Jan 30, 2024

CamilleBeau commented Jan 30, 2024

maximemulder Feb 2, 2024

CamilleBeau Feb 6, 2024

CamilleBeau commented Feb 6, 2024

maximemulder Feb 8, 2024

kongtiaowang Feb 8, 2024

driusan Feb 8, 2024

maximemulder Feb 8, 2024 •

edited

Loading

driusan Feb 8, 2024

CamilleBeau Feb 8, 2024

CamilleBeau commented Feb 8, 2024

kongtiaowang Feb 8, 2024 •

edited

Loading

CamilleBeau Feb 8, 2024

[candidate] Create Sex table and remove Sex enum #9025

[candidate] Create Sex table and remove Sex enum #9025

Conversation

CamilleBeau commented Jan 29, 2024 • edited Loading

Brief summary of changes

Testing instructions (if applicable)

Link(s) to related issue(s)

CamilleBeau commented Jan 29, 2024

kongtiaowang commented Jan 30, 2024

CamilleBeau commented Jan 30, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

CamilleBeau commented Feb 6, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

maximemulder Feb 8, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

CamilleBeau commented Feb 8, 2024

kongtiaowang Feb 8, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

CamilleBeau commented Jan 29, 2024 •

edited

Loading

maximemulder Feb 8, 2024 •

edited

Loading

kongtiaowang Feb 8, 2024 •

edited

Loading