Location Module - visual interface

[kvesik]

Notes for when this is ready to be implemented:

• Sophie is creating mockups (20230329) of highlighting locations on body using both shading and outlining.
• we'll have different sets of locations (and related image files) associated with different theoretical frameworks, that users can change/restrict through settings
• how to choose sub-areas: either
  • Maggie's idea of clicking multiple times on the same location to drill down (or up) through specificity, with a global setting to choose which end of specificity continuum to start at, or
  • earlier discussion of clicking on a region, then choosing which level of specificity to use from a menu that pops up (and again, could have a global setting that will auto-select either highest or lowest as default)

[kvesik]

Once this is implemented, decide whether it should be available for handpart selection as well.

[kchall]

Just a reminder that we have several .ppt files with thoughts about how to progress through different levels. We also have a full set of locations in each of three colours (yellow, violet, and green), if that helps us visually represent things.

[kvesik]

Just a reminder that we have several .ppt files with thoughts about how to progress through different levels. We also have a full set of locations in each of three colours (yellow, violet, and green), if that helps us visually represent things.

From 20231016 meeting...

• I asked re above: Was there a final decision on how to progress through the various options? Is it up to me? and what files other than Update Locations/Navigating_Locations.pptx discuss these options?
• KCH says: Navigating_Locations.pptx is the most recent/comprehensive, so just use that as reference. BUT otherwise, no final decisions were made. I will figure out what seems to be most effective/efficient, and ask to meet with Kathleen separately if I want to clarify anything or discuss pros/cons of various options.

[kvesik]

@kchall I have some questions about formatting of location option text, as well as some misalignments between which text vs images are available.

I've made a list of my specific questions below, but if you're curious there's detailed info in this spreadsheet.

Currently the way that the images are named doesn't precisely line up with the text options, so once we've got this sorted out I'm just going to batch-rename the images so that they're easier to link up with the text options as the user specifies the location module.

1. In general, all of the locations that are not specified for contra/ipsi are identified in the text as singular (whether they're contiguous or discrete). Eg, "Jaw" refers to both (contiguous) sides of the jaw and "Ear" refers to both (discrete) ears. And when they are specified for contra/ipsi, they remain singular; eg "Jaw - contra" or "Ear - ipsi". My assumption is that this is all ok and as intended, but just wanted to confirm.

2. (1) above means that even if there are two separate sides of a location shown in the image (eg, both ears are highlighted) the text alongside would still show singular (eg, "Ear"). Right?

3. Here is a list of images that I have from Sophie, that do not currently have corresponding text options in the location tree. Please confirm whether I should add a text option for each of these into the location tree, and if so whether or not you like my proposed wording. I have the feeling that the sub-hand locations don't need separate entries in the location tree, but wanted to confirm just in case.

   • Eyelids [both upper and lower] of the contra/ipsi eye. I propose "Eyelid - contra" and "Eyelid - ipsi"
   • Corners [both contra and ipsi] of the mouth. I propose "Corner of mouth"
   • Contra/ipsi whole hands. I propose "Whole hand - contra" and also ipsi
   • Contra/ipsi hand minus fingers. I propose "Hand minus fingers - contra" and also ipsi
   • Contra/ipsi heel of hand. I propose "Heel of hand - contra" and also ipsi
   • Contra/ipsi fingers and thumb. I propose "Fingers and thumb - contra" and also ipsi
   • Contra/ipsi thumb. I propose "Thumb - contra" and also ipsi
   • Contra/ipsi fingers. I propose "Fingers - contra" and also ipsi
   • Contra/ipsi fingers 1, 2, 3, 4. I propose "Finger 1 - contra" and also ipsi, plus fingers 2, 3, 4 as well
   • Contra/ipsi between fingers. I propose "Between fingers - contra" and also ipsi
   • Contra/ipsi between thumb and Finger 1. I propose "Between thumb and Finger 1 - contra" and also ipsi
   • Same as above but for all the other combinations too (fingers 1+2, 2+3, 3+4).

4. Here is a list of text options in the location tree, that currently do not have corresponding images from Sophie. Please confirm whether we need images of these locations. Some of them will be easy for me to do (eg buttocks), but the others will need a more skilled artist.

   a. Upper teeth
   b. Lower teeth
   c. Tongue
   d. Buttocks - contra & ipsi
   e. Selected fingers and thumb (is this something we'd eventually hope to do progammatically? ie, highlight the specific fingers that have been marked as "selected" elsewhere in the software)
   f. Selected fingers (same question as for (e))

Thanks!

[kvesik]

Notes from 20240311 meeting:

• add all missing texts that have images (including sub-hand locations)
• for inside of mouth texts, ok to not have these images right now... but could temporarily use just the "mouth" image in place of tongue, upper/lower teeth, etc
• for buttocks, make a L & R image
• for selected fingers (and thumb, if applicable), the idea of programmatically combining finger images is a good thing to eventually do (but not a top priority)

[kchall]

Right...and:

• For (1) and (2), yes, fine to have the generic label always be singular, even if the image is showing both elements highlighted.

[kvesik]

Notes from 20240318 meeting:

• make sure ordering and hierarchy (indenting?) of locations in right-click menu make sense; these should parallel the structure in the combobox
• single L-clicks just cycle through images (from high to low or low to high, as per settings) without selecting/adding to list
• summarize any location name text changes for Kathleen
• re-upload SVGs to google drive once they are ready

Notes from 20240212 meeting:

• for user interaction, consider including "with divisions" in right-click list? (n/a; type D on keyboard)
• for user interaction, consider a global settings that determines whether divisions are shown or not? (n/a; type D on keyboard)

From 20240205 meeting:

• go for simplicity / intuitiveness on visual interface. if users want shortcuts they can use the text search

[kvesik]

@kchall at last week's meeting I asked you how we should deal with multiple sub-trees when single-left-through the regions on the location diagram. The decision we came to was that no matter whether the user has the cycle order set to "small to large" or "large to small", we should cycle through each subtree in that order. However, I'm realizing that although this makes intuitive sense for large to small, it's absolutely terrible for small to large.

For example, consider the click location below (on the left hand). The region options available here are shown in the R-click menu:

For "large to small", we would simply cycle through from top to bottom. But for "small to large", the order would be (each subtree from top to bottom, from smallest to largest within each subtree):

• whole hand contra
• hand minus fingers contra
• hand minus fingers
• fingers and thumb contra
• finger 1 contra
• finger 1
• fingers contra
• fingers
• betw thumb and f1 contra
• betw thumb and f1
• betw fingers contra
• betw fingers
• fingers and thumb

This seems completely unintuitive to me. Just clicking through and testing it, it doesn't feel like there's any logic to the order at all (even though there is, of course).

I propose that we either:
(a) only allow the large-to-small cycle, or
(b) have the small-to-large cycle go up from the bottom of the entire menu to the top, rather than bottom to top of each submenu (where the submenus are traversed from top to bottom). This is much more intuitive than the way we originally discussed.

What do you think?

[kchall]

@kvesik Ah, good point. Let's go with (b), which is basically the 'same' as we originally planned, but also flipping the order of the sub-menus, right? i.e., bottom to top of each sub-menu, with sub-menus also being traversed from bottom to top?

[kvesik]

@kchall If a user selects a location from the list for which there is no image (eg mouth-interior locations, or anything involving selected fingers), the location dialog just shows the default non-highlighted image (whereas normally a selection from the list would show a corresponding highlighted image), and throws an error in the background because it's looking for an image with a particular name and there isn't one.

Would you like me to:

• (a) leave it as is (so, no highlighted region shown) and just make sure to catch the error in the background? or
• (b) show some sort of more appropriate default image (eg, "mouth" for any mouth-interior locations, or "hand" for any selected finger locations)?

[kvesik]

Note for documentation - methods for interacting with Location images.

Right-click

• User right-clicks anywhere on body image
• A context menu appears, showing options for location regions that are available at the click point
• User selects one of the options and it is added to the selected locations list on the right-hand side of the dialog

Left-click

• User left-clicks anywhere on body image
• The smallest (or next-smallest, if a region is already highlighted) region available at that click point is highlighted but NOT added to the selected locations list
• "smallest" here means "lowest in the menu that would come up if the user had right-clicked"
• setting can be switched between smallest-->largest vs largest-->smallest in the Location tab of the Settings dialog in the main window

Keyboard input

When focus is on the location image...

• pressing the "S" key selects the current image and adds it to the selected locations list on the right-hand side of the dialog
• pressing the "D" key toggles between current image with/without divisions (if available)

[kchall]

@kchall If a user selects a location from the list for which there is no image (eg mouth-interior locations, or anything involving selected fingers), the location dialog just shows the default non-highlighted image (whereas normally a selection from the list would show a corresponding highlighted image), and throws an error in the background because it's looking for an image with a particular name and there isn't one.

Would you like me to:

• (a) leave it as is (so, no highlighted region shown) and just make sure to catch the error in the background? or
• (b) show some sort of more appropriate default image (eg, "mouth" for any mouth-interior locations, or "hand" for any selected finger locations)?

Good question! I'm thinking that maybe this is a good place to use one of our different-coloured highlighted versions. So e.g. do (b) but with the purple version of the mouth / hand picture to suggest that this is not quite the right image?

[kvesik]

@kchall Here's a summary of the non-code changes I made (or am about to make) as part of the Location images implementation. Some may be more relevant for you, others for a future RA who is working on the outstanding drawing-related tasks (see eg issue #337).

Changes to Location text descriptions

• "Nostrils" → "Nostril" (even when showing/referencing both); this is in line with all other regions, which only ever use the singular form
• Added several Location regions to the tree, for which we had images but hadn't originally had text:
  • "Whole hand - contra" and "Whole hand - ipsi"
  • "Hand minus fingers - contra" and "Hand minus fingers - ipsi"
  • "Heel of hand - contra" and "Heel of hand - ipsi"
  • "Fingers and thumb - contra" and "Fingers and thumb - ipsi"
  • "Thumb - contra" and "Thumb - ipsi"
  • "Fingers - contra" and "Fingers - ipsi"
  • "Finger 1 - contra" and "Finger 1 - ipsi", as well as for Finger 2, 3, 4
  • "Between fingers - contra" and "Between fingers - ipsi"
  • "Between Thumb and Finger 1 - contra" and "Between Thumb and Finger 1 - ipsi", as well as between Fingers 1&2, 2&3, 3&4
  • "Eyelid - contra" and "Eyelid - ipsi"
  • "Corner of mouth" (previously we had only ipsi and contra)

Changes to Location image .svg filenames

Renamed/reformatted to ensure that there is an unambiguous relation between the region names in the .svg files, the descriptive region names (eg that the user interacts with), and the .svg filenames themselves (this relation is explained/exemplified below).

Changes inside Location image .svg files

Sophie originally created a large collection of drawings, each showing an individual region of the body highlighted in yellow, violet, or green. I made two changes to the way that the regions were labeled in these .svg files:

• I renamed/reformatted many of the internal region labels to ensure that there is an unambiguous relation between the region names in the .svg files, the descriptive region names (eg that the user interacts with), and the .svg filenames (this relation is explained/exemplified below).
• I added region labels to each .svg file. I made sure that every .svg file contains labels for every Location region that's visible in that view; this is the method by which the right-click menu (or the left-click cycle through regions) works when a user is interacting with the Location images. If a region label is missing from inside the .svg, then that region will not be available for the user to interact with on the image in the Location module.

Naming relation

In order to have an efficient way to convert back and forth between the region names in the .svg files, the descriptive region names (eg that the user interacts with), and the .svg filenames, there needs to be an unambiguous relation between each pair of these elements. The updated naming/formatting convention I implemented involves the following:

• Descriptive region names (eg that the user interacts with) all have their first word capitalized, and if they refer to ipsi or contra side, end with " - " followed by "ipsi" or "contra". They might also have a forward slash in the name somewhere, whether flanked by spaces ("Septum / nostril area") or not ("Abdominal/waist area"). The spaces around the slashes aren't consistent, but for the sake of avoiding a bunch of backward compatibility code I just left them as they were.
• .svg filenames are converted from descriptive region names by replacing spaces " " with underscores "_", and slashes "/" with text placeholders "-slash-". If the image refers to one specific side, it's prepended with "Right_" or "Left_". Each image also has one or more suffixes:
  • "_with_Divisions" (optional)
  • "-yellow" (or violet, or green)
• Internal region name labels (that is, the id attribute of the g element: <g id="...">) inside the .svg files are exactly the same as the .svg filenames, but without the "-yellow" (or whatever colour) tag at the end.
  • If a region is split into contra/ipsi (left/right), then it gets two layers of tags: one general one (eg just "Arm") and one specific one (eg "Right_Arm"). However, because we can't have duplicate IDs (otherwise the first one gets overwritten by the second), the general tag for the left region will have a 0 (zero) tagged onto the end of the string. That wording is kind of confusing, so here's an example:

<g id="Arm" data-name="Arm">
    <g id="Right_Arm" data-name="Right Arm">
      <path class="cls-100" d="M449.2,799.63c2.27-5.33,4.71-10.7,7.42-16.23,9.11-18.62,17.68-37.42,24.73-56.92,6.43-17.8,14.46-37,17.47-55.75a78.06,78.06,0,0,0,1.08-15.27c-.57-15.22-2.31-29.75-2.34-44.92,0-19.74,2.77-39.3,4.11-59v-1.76h.06l-.66-2.82c-2.2-9.49-2.47-19.57-3.89-29.23L494,496.43l-.7-7.91c-3.89-43.83,27.55-81.58,39.83-122.24.33.23-1.91,5.81-2.19,6.54-.89,2.33-1.9,4.61-2.93,6.87l-1.62,3.53-1.62,3.53-1.62,3.53-1.62,3.54c-2.21,4.84-4.79,9.76-8.87,13.28-7.67,6.6-18.2,7.69-27.91,7.94-15.83.4-31.86-1.78-47.29-5.2-1-.22-7.18-2.14-7.74-1.11-6.18,11.86-6.8,27.45-8.08,40.55a288.26,288.26,0,0,0-1.1,42.88l4.13,77.92c1.13,21.27,5.51,43.47,2.95,64.56-1.63,13.48-13.18,25.21-18.41,38.06-12.16,29.82-13.14,62.51-16.18,94.14a291.19,291.19,0,0,1-17.89,76.22,22,22,0,0,1,4.12.6c8.45,2.13,17,3.83,25.49,5.78,4.25,1,8.49,2,12.69,3.18,3.42,1,7.79,1.88,10.09,4.88a21.78,21.78,0,0,1,2.36,4,59.52,59.52,0,0,1,2.64-16C437.91,829.21,442.83,814.63,449.2,799.63Z"/>
    </g>
  </g>
  <g id="Arm0" data-name="Arm">
    <g id="Left_Arm" data-name="Left Arm">
      <path class="cls-100" d="M846.8,799.63c-2.27-5.33-4.71-10.7-7.42-16.23-9.11-18.62-17.68-37.42-24.73-56.92-6.43-17.8-14.46-37-17.47-55.75a78.06,78.06,0,0,1-1.08-15.27c.57-15.22,2.31-29.75,2.34-44.92,0-19.74-2.77-39.3-4.11-59v-1.76h-.06l.66-2.82c2.2-9.49,2.47-19.57,3.89-29.23l3.15-21.3.7-7.91c3.89-43.83-27.55-81.58-39.83-122.24-.33.23,1.91,5.81,2.19,6.54.89,2.33,1.9,4.61,2.93,6.87l1.62,3.53,1.62,3.53,1.62,3.53,1.62,3.54c2.21,4.84,4.79,9.76,8.87,13.28,7.67,6.6,18.2,7.69,27.91,7.94,15.83.4,31.86-1.78,47.29-5.2,1-.22,7.18-2.14,7.74-1.11,6.18,11.86,6.8,27.45,8.08,40.55a288.26,288.26,0,0,1,1.1,42.88l-4.13,77.92c-1.13,21.27-5.51,43.47-2.95,64.56,1.63,13.48,13.18,25.21,18.41,38.06,12.16,29.82,13.14,62.51,16.18,94.14a291.19,291.19,0,0,0,17.89,76.22,22,22,0,0,0-4.12.6c-8.45,2.13-17,3.83-25.49,5.78-4.25,1-8.49,2-12.69,3.18-3.42,1-7.79,1.88-10.09,4.88a21.78,21.78,0,0,0-2.36,4,59.52,59.52,0,0,0-2.64-16C858.09,829.21,853.17,814.63,846.8,799.63Z"/>
    </g>
  </g>

Some examples:

1. • descriptive name: Septum / nostril area
   • filename: Septum_-slash-_nostril_area-yellow.svg or Septum_-slash-_nostril_area_with_Divisions-yellow.svg
   • region label (ID tag) inside .svg file: Septum_-slash-_nostril_area
2. • descriptive name: Corner of mouth - contra
   • filename: Left_Corner_of_mouth-yellow.svg
   • region label (ID tag) inside .svg file: Corner_of_Mouth0 and Left_Corner_of_Mouth
3. • descriptive name: Thumb - ipsi
   • filename: Right_Thumb-yellow.svg
   • region label (ID tag) inside .svg file: Thumb and Right_Thumb

Placeholder images

There are a five Location options for which we don't have images. The first three because we don't have an inside-of-mouth view, and the last two because we don't have any behaviour or related logic implemented at all for "selected fingers."

• Tongue
• Upper teeth
• Lower teeth
• Selected fingers
• Selected fingers and thumb

There are a few things to note about these five Locations:

1. They have placeholder images:
   • "Mouth" stands in for "Tongue"
   • "Teeth" stands in for "Upper Teeth"
   • "Teeth" stands in for "Lower Teeth"
   • "Fingers" stands in for "Selected fingers" (note only the both-handed version is available, no contra/ipsi options)
   • "Fingers and thumb" stands in for "Selected fingers and thumb" (note only the both-handed version is available, no contra/ipsi options)
2. The placeholder images have regions highlighted in violet instead of yellow, so it's clear to the user that there's something different about these ones.
3. In order for the placeholder images/locations to come up in the image right-click menu and the left-click cycle, all (front-view) .svg files had to have internal region name labels (that is, the id attribute of the g element: <g id="..."> as discussed above) added for these Locations. However, since they don't have their own highlighted region drawings yet, the region specifications were just copied directly from the placeholder regions.
4. In the "yellow_20240703version" folder, there is a .txt file (named similarly to the corresponding .svg) for each of these regions, flagging that it's a placeholder.

Re-organization of Location images in Google drive

In SLP-AA/Drawings/Updated Locations Sophie originally created folders called:

1. "Highlighted Locations", which contained subfolders for yellow, green, and violet version .svg files for all Location regions.
   • This is now renamed to "Highlighted Locations - original versions from Sophie" and the updated versions are in a folder called Highlighted Locations - updated versions from Kaili
2. "Views", which contained just a front, a side, and a back view file without any region highlighting.
   • This is now renamed to "Views - original versions from Sophie" and the updated versions are in a folder called Views - updated versions from Kaili

[kchall]

@kvesik Awesome, thanks for keeping track of these changes and documenting them so clearly -- all looks good to me so far!