GSoC Idea: Implement user roles/permissions and OpenID

[vchrombie]

Right now, SortingHat only supports one type of user: the admin user. We would like to have different roles, so basic users don't have all the rights to do evil things when using the service. Additionally, we want to integrate OpenID as a secure protocol for users to sign-in.

SortingHat is the tool that we use to manage identities data in GrimoireLab. As individuals in a project can have different identities - several usernames or email addresses - this tool allows creating unified profiles of them. Then, the platform will use this information to generate accurate results of the activity of these participants.

SortingHat started as a command-line tool but after some years, we saw its potential and we decided to create a new version, this time as a service. This new version provides a new GraphQL API to operate with the server and a UI web-based app, that replaces Hatstall, the old UI for SortingHat.

Although the development of it is in its later stage and it will be ready soon for the stable version of the platform, there are many good ideas that we will like to incorporate. Some of them were selected for GSoC 2021.

The aims will require working with:

• Identify and define roles for SortingHat.
• Implement permissions for the different kinds of roles.
• Add support for OpenID.
• (Extra stretch goal) Roles and user management using the UI.

The aims will require generating code in Python for Django and the GraphQL API, and for the web app (generated with Vue.js and Vuetify).

• Difficulty: Medium
• Requirements: Interest in software analytics. Python programming. JavaScript programming. SQL knowledge. Willingness to understand GrimoireLab internals.
• Recommended: Experience with Python, JavaScript, UI development, GraphQL, Django, and Vue.js would be convenient but can be learned during the project.
• Mentors: @sduenas @evamillan @mafesan

Microtasks

For becoming familiar with GrimoireLab, you can start by reading some documentation. You can find useful information at:

• GrimoireLab Tutorial
• SortingHat: Wizardry on Software Project Members
• The GitHub repositories hosting the tools (chaoss/grimoirelab-perceval, chaoss/grimoirelab-elk, chaoss/grimoirelab-*...).

Once you're familiar with Grimoirelab, you can have a look at the following microtasks.

• Microtask 0:
  Download PyCharm and get familiar with it (for instance, you can follow this tutorial).

• Microtask 1:
  Set up a dev environment to work on GrimoireLab. Have a look at chaoss/grimoirelab-sirmordred - Getting-Started.md.

• Microtask 2:
  Execute micro-mordred to collect, enrich and visualize data from Git repositories.

• Microtask 3:
  Based on the elasticsearch documents produced by micro-mordred and source code of chaoss/grimoirelab-elk, try to answer the following questions:

  • What is the meaning of the JSON attribute author_id?
  • What is the meaning of the JSON attribute author_org_name?
  • What is the meaning of the JSON attribute author_uuid?
  • What is the meaning of the JSON attribute author_domain?
  • What is the meaning of the JSON attribute uuid?
  • What is the meaning of the JSON attribute utc_commit?
  • What is the meaning of the JSON attribute origin?

• Microtask 4:
  Set up the developer environment of SortingHat (muggle branch).

  NOTE: The sortinghat muggle branch is a WIP branch. As of now, it doesn't work with the core of the GrimoireLab platform but we hope to have it ready soon.

• Microtask 5:
  Create a sample profile with different identities and enrollments using the SortingHat UI.
  

• Microtask 6:
  Using the SortingHat GraphQL Console, create a query that fetches the data (identities, enrollments) of an individual profile.

• Microtask 7:
  Create a script that can parse the gitdm developer affiliation files and load the data in a SortingHat database using GraphQL.

• Microtask 8:
  Improve the visualization of the individualCards component. You need not send a PR, please update the work in your personal fork.

• Microtask 9:
  Submit a PR to any of the GrimoireLab components to increase the test coverage of one or more files of the source code.

• Microtask 10:
  Submit at least a PR to one of the GrimoireLab repositories to fix an issue, improve the documentation, etc. Some good-first-issues are:

  • Command line client displays RuntimeError traceback grimoirelab-perceval#692
  • Improve log messages grimoirelab-perceval#691
  • Missing value in GitHub Events CSV entry grimoirelab-elk#951
  • Describe Twitter Schema grimoirelab-elk#936
  • Update schemas to latest format grimoirelab-elk#803

[AllMight2099]

Hey there!
My name is Nishanth and I'm a sophomore from IIT Roorkee. I interned as a Django backend developer for a couple of months and had actually worked on a similar issue at my internship, so it would be really interesting to work on a complex version of this. I have also worked on multiple Javascript projects, one of them being the front-end portion of our campus group's website that hosts machine learning competitions (using React), so I do have a strong understanding of Javascript too!
I've also set up GrimoreLabs and I'm fairly familiar with its workings, but I would love to get a little bit of guidance too. Looking forward to working with you guys!

[Riyabelle25]

Hi guys!
I'm Riya Elizabeth John, a sophomore from IIT Roorkee, India. I found CHAOSS back in February and both Augur and GrimoireLabs really piqued my interest, so I followed the set-up and got familiar with the tutorials.
Python is my primary programming language and I have worked with and contributed to Django based applications as part of my institute's developer group. Really excited about seeing the idea proposals this time, and hope to learn and make some valuable contributions😄!

[VSevagen]

Hi there !
My name is Sevagen and I'm from Amrita Vishwa Vidyapeetham, Amritapuri Campus. I'm also a member at amFOSS, a student-driven community based in our campus. I've been looking around the idea list and found this one pretty interesting. I started with the micro tasks and will try my best to complete them as soon as possible.

[vchrombie]

Hi everyone, thanks for your interest in applying for this idea. You can start working on the microtasks to get a better idea of the project. Let us know if you have any doubts. 🙂

For all students interested in this idea, please comment on this issue to get in touch with the mentors. This is the main communication channel.

[vchrombie]

Hi @AllMight2099 @Riyabelle25 @VSevagen
I hope you started working on the microtasks. As you might know, you have to submit a proposal before the GSoC deadline. You are also expected to attempt at least one microtask for considering your application.

The main reason behind the microtasks is, these tasks will give a good minimum understanding of the Sorting Hat tool as well as the GrimoireLab platform as a whole. It will be really helpful for writing your proposal.

If you haven't started working on the microtasks yet, I would suggest you start asap. You can create a github repository for storing the microtasks and you can open issues in that repo for asking doubts or reviewing the tasks.

Thanks.

[AllMight2099]

Hey @vchrombie!
I'm done with MIcro Task 1, had an issue with my elasticsearch docker container, where Sorting Hat wasn't able to access the container and I kept getting this error

HTTPConnectionPool(host='localhost', port=9200): Max retries exceeded with url: / (Caused by ProtocolError('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')))

Considering that I've understood the codebase, I just need to deploy micro-mordred. But the docker error isn't allowing me to proceed

[SourabhSaraswat-191939]

Hi there !
Myself Sourabh Saraswat and I am pursuing my Under-graduation in Computer Science and Engineering from Central University Of Haryana . I gone through some tutorial and I've been looking around the idea list and found this project idea interesting . I already started working on micro tasks . Hope that i will complete it as soon as possible .

[vchrombie]

Hi @AllMight2099

I'm done with MIcro Task 1, had an issue with my elasticsearch docker container, where Sorting Hat wasn't able to access the container and I kept getting this error

Did you mean Sir Mordred?

Can you please confirm if the elasticsearch service is reachable? You can use curl -XGET -k <elasticsearch-url> to check it.

It would be great if you share some more information like the steps and the docker-compose file to have a closer look.

[AllMight2099]

Yeah, I meant micro-mordred actually. This was the response that I got from curl
curl: (56) Recv failure: Connection reset by peer

And my docker-compse file for elastic search was this:

    mariadb:
      image: mariadb:10.0
      expose:
        - "3306"
      environment:
        - MYSQL_ROOT_PASSWORD=
        - MYSQL_ALLOW_EMPTY_PASSWORD=yes
    elasticsearch:
      image: bitergia/elasticsearch:6.8.6-secured
      command: elasticsearch -Enetwork.bind_host=0.0.0.0 -Ehttp.max_content_length=2000mb
      ports:
        - 9200:9200
      environment:
        - ES_JAVA_OPTS=-Xms2g -Xmx2g
    kibiter:
      restart: on-failure:5
      image: bitergia/kibiter:secured-v6.8.6-3
      environment:
        - PROJECT_NAME=Demo
        - NODE_OPTIONS=--max-old-space-size=1000
        - ELASTICSEARCH_USER=kibanaserver
        - ELASTICSEARCH_PASSWORD=kibanaserver
        - ELASTICSEARCH_URL=["https://elasticsearch:9200"]
        - LOGIN_SUBTITLE=If you have forgotten your username or password ...
      links:
        - elasticsearch
      ports:
        - 5601:5601```

[AllMight2099]

Okay, so, I installed the grimorelabs using docker this time and it works. Sir Mordred works fine.
I've moved onto the next task now.

If anyone faces a similar issue with Elasticsearch, try installing grimore labs using the "Only Docker" approach

[vchrombie]

Yeah, I meant micro-mordred actually. This was the response that I got from curl
curl: (56) Recv failure: Connection reset by peer

The elasticsearch is not reachable. You should find something useful in the logs while running the docker-compose for elasticsearch. It would be great if you can check the logs once. Most of the time, it is a low memory issue or something should be wrong with the configurations, troubleshooting. We can meet once if you want to dig more to find the cause of the problem.

Okay, so, I installed the grimorelabs using docker this time and it works. Sir Mordred works fine.
I've moved onto the next task now.

If anyone faces a similar issue with Elasticsearch, try installing grimore labs using the "Only Docker" approach

Sorry @AllMight2099, but I wouldn't suggest this. The docker solution is for the people who are interested in using GrimoireLab but are not interested in its source code. It would be better if you can run micro-mordred using the setup described in the microtask-1.

[AllMight2099]

Hey @vchrombie! Sorry about the delay, had a few tests and couldn't come online. I fixed the issue, it was the low memory usage issue itself. Had an issue with SearchGuard too, I tried fixing that, but couldn't. Had to use a docker-compose without Search Guard, but everything's up and running now, I'll put up my microtask submission by tomorrow and then I'll get started on the proposal

[vchrombie]

No problem @AllMight2099

Had to use a docker-compose without Search Guard, but everything's up and running now, I'll put up my microtask submission by tomorrow and then I'll get started on the proposal

Even that works, but make sure you change the es endpoints in the setup.cfg file if you are using without searchguard.

[es_collection]
url = http://localhost:9200

[es_enrichment]
url = http://localhost:9200

The setup.cfg has all the configurations related to es, kibiter, sortinghat, etc. So, please check if the configurations are correctly set.

[vchrombie]

Hi everyone, the student application period has started and the deadline is 13 April 2021, 18:00 UTC. GSoC Timeline

Please continue working on the proposal and complete as many microtasks as possible. Please let us know if you need any help with doubts or reviewing the microtasks.
Thanks!

[SourabhSaraswat-191939]

@vchrombie is their any restriction to select a provider in OpenID or i can select Google as per my priority for OpenID connect .

[SourabhSaraswat-191939]

@vchrombie , i have one Doubt that is as per this GSoC idea we have to implement roles/permissions for users , but django already provides us this feature in his admin panel in the form of groups and permissions . So what the work we have to do during this idea .
Does we have to make this roles/permissions from scratch by designing database , logic and all? Or we have to work on django roles/permissions to customize it ?
Can you please explain it .

[sduenas]

@SourabhSaraswat-191939 That's up to you and it's part of your proposal. If we can use django functionalities, that will be a very good option but it doesn't mean it's the only one. You can come up with an alternative that works better than Django's roles and groups for our case. You just need to explain what your solution is better than others.

Keep in mind that the idea is to have those roles should restrict what operations users can do. Also, those roles will be very important when they are integrated with OpenID.

[SourabhSaraswat-191939]

Ok @sduenas ,
I will prefer to go with most optimize and secure approach to come with the best solution for this .

[vchrombie]

Thanks to everyone who showed interest in applying for this idea and worked on making a proposal and the microtasks. It was great working with you.

As the final steps, please make sure you submit your proposal on the Google Summer of Code website and you also need to open a PR adding your name and details to the GSoC-interest.md file in order to qualify as an interested candidate. Both have to be completed before the deadline mentioned on the GSoC website.

Thanks once again! All the best.

[vchrombie]

Thanks everyone for participating in this idea! Unfortunately, no students have been selected for this idea despite the good applications.

The decision process was divided into evaluation and discussion. In the evaluation phase, the mentors checked the proposals based on different dimensions (e.g., clarity, soundness, complexity, microtasks). In the discussion phase, the mentors shared their views on the proposals and democratically made a decision.

But, I believe that the time, interest, and contributions to GrimoireLab of @VSevagen, @SourabhSaraswat-191939, and @AllMight2099 have been remarkable, and it would be great if you would like to keep contributing to GrimoireLab. At the same time, I totally understand your mood right now.

If you have any questions, comments, or concerns about the selection process, feel free to write here or send an email to venu@bitergia.com. Thanks!

This issue is going to be closed on Friday.