-
-
Notifications
You must be signed in to change notification settings - Fork 620
Google Season of Docs 2021 Case Study
-
Organization or Project: https://github.com/pytorch/ignite, https://pytorch-ignite.ai,
-
Organization Description: High-level library to help with training and evaluating neural networks in PyTorch flexibly and transparently.
-
Authors: Victor (@vfdev-5), Sylvain (@sdesrozis), Taras (@trsvchn)
Our old documentation is not well-organized and needs a restructure, refresh and an update. We provide a large amount of information that finally does not help to understand basics and the power of the tool.
In GSoD project we are proposing to restructure our docs on our new website following Diataxis framework.
Project plan is the following:
- Audit the existing documentation
- Create a friction log of the current documentation for the use cases
- Use the friction log as a guide to rewrite the documentation and examples keeping only essential and relevant information
- Incorporate feedback from documentation testers (volunteers in the project) and the wider PyTorch-Ignite community.
It was decided within maintainer's team to apply to GSoD and create a proposal. Our proposal was inspired by official proposal template and official examples. Once the proposal was written it was then validated by the team and incorporated the feedbacks.
Our budget estimation was based on other projects examples and we assumed that ~1100$/month for a non-fulltime work can be correct. In parallel to tech writer's work we organized our new website development that was funded outside of Season of Docs and helped to make both projects to be synchronized and move forward together.
Technical writer: Priyansi (@Priyansi) Volunteers: Sylvain (@sdesrozis), Taras (@trsvchn) and Victor (@vfdev-5).
How did you find and hire your technical writer?
We announced on our media channels that our project is participating in GSoD and we got more than 10 applications. We shortlisted 4 of them and did a video call to talk with candidates. It was a hard decision to choose between two finalists, their applications and performance have been very strong. We finally selected Priyansi as our tech writer.
How did you find other volunteers or paid participants? What roles did they have? Did anyone drop out?
The role for volunteers is mainly explain our project, answer questions and review the work. Three members of our core team participated as volunteers.
What did you learn about recruiting, communication, and project management?
This was an unique opportunity for us to put ourselves in recruiter's shoes. Recruiting process was interesting and very important to make a well-rounded hiring decision. We learnt the importance of non-technical skills for effective communication, teamwork and for project long-term success in common.
Give a short overview of the timeline of your project (indicate estimated end date or intermediate milestones if project is ongoing).
The tasks were scheduled in priority according to Diataxis framework recomendations. This allowed us to cover a significant space of the community needs, targeting end-users of various level of experience from beginner to advanced.
The project is almost complete and most of intermediate milestones were achieved in time. For the overdue tasks there were no critical delays. The following goals were successfully achieved:
- May 2021 - Onboarding and community bonding
- June/July 2021 - Library documentation audition and friction log preparation
- August/Sept/Oct/Nov 2021 - Work on Tutorials
- August/Sept/Oct/Nov 2021 - Work on How-to guides
- Nov/Dec 2021 - Finalizing open tasks and incorporating early user feedback
For more details, please see the complete timeline.
What was created, updated, or otherwise changed? Include links to published documentation if available.
The technical content of our new website (https://pytorch-ignite.ai/getting-started/) is the result of this GSoD2021 work.
-
Tutorials
- Getting Started
- Distributed Training on CPUs, GPUs or TPUs
- Collective Communication with Ignite
-
How-to Guides
- How to install PyTorch-Ignite
- How to convert pure PyTorch code to Ignite
- How to do time profiling
- ... other 11 how-to guides
Were there any deliverables in the proposal that did not get created? List those as well.
Unfortunately, certain tasks are not yet finished: https://github.com/pytorch-ignite/examples/issues and https://github.com/pytorch-ignite/pytorch-ignite.ai/issues/67. We think to finish these tasks by the end of the year.
What metrics did you choose to measure the success of the project? Were you able to collect those metrics? Did the metrics correlate well or poorly with the outcomes you wanted for the project? Did your metrics change since your proposal?
We proposed to measure the success of the project by the following metrics:
- Better SEO for our documentation
- Decrease in number of issues raised for topics covered in the documentation
- Increase in number of documentation contributions
Better SEO for our documentation: We are gathering the information with Google Analytics for our old and new websites. We will analyze user's behaviour in details once we accomplished the project.
Decrease in number of issues raised for topics covered in the documentation: We are tracking such issues and questions on GitHub and Discord, where users can ask all their questions. We do not have much questions and few recent questions revealed a lack in the current documentation. We are incorporating the feedback.
Increase in number of documentation contributions: We participated in Hacktoberfest and in PyData Global Mentored Sprint this year where we incited participants to contribute to the documentation and we got 8 community PRs merged for https://github.com/pytorch-ignite/examples repository. Prior to this GSoD work, we had very small number of documentation contributions.
We also checked our results if they answer the feedback we gathered from our first community survey (see below).
As an additional metric, there is a plan to create the second community poll about our finalized documentation and compare the feedbacks with the first poll we did in the beginning.
What went well? What was unexpected? What hurdles or setbacks did you face? Do you consider your project successful? Why or why not? (If it's too early to tell, explain when you expect to be able to judge the success of your project.)
Priyansi's integration into the team went perfectly. From the beginning she proposed a workplan that we agreed all together. She followed then the plan with minor changes discussed during our weekly meetings. All the work done is of good quality and the team approved it.
In addition to the technical writing, we agreed that she could also help with community management and communications. All these additional roles were done successfully.
The scope of the project was quite heterogeneous and sometimes domain-specific, some tasks required deeper understanding/investigation of the subject, methods and tooling resulting in a more detailed review process and code validation. These include writing tutorials on more advanced topics like distributed training, reinforcement learning, etc.
Although the work is not fully completed, the project can be considered successful, due to the quality of the technical documents provided and the organization carried out. We will be able to quantify the success with respect to proposed metrics in a few months.
In 2-4 paragraphs, summarize your project experience. Highlight what you learned, and what you would choose to do differently in the future. What advice would you give to other projects trying to solve a similar problem with documentation?
It is our first application and participation in Google Season of Docs and our project experience is very positive.
During the project we learned how to recruit a suitable technical writer, how to onboard and integrate him/her into the team. We also learned how to manage our and his/her activity as one of the difficult points is also managing the team's time to be available to technical writer's questions, reviewing his/her work and other organizational things. Due to that we would reallocate differently the budget between technical writer and volunteers work in order to have at least one or two mentor(s) more available to work with the technical writer.
In open-source scene project's sustainability is an important factor to take into account. Converting the technical writer to the project's documentation maintainer is a hard question. In our case, we are try to make that possible such that it could be another success of our GSoD program.
| Feedback | How has it been addressed? | Any WIP or future plans? |
|---|---|---|
| Users were not satisfied with the flow of the documentation. | We have restructured our website into 4 segments based on Diataxis that clearly defines a flow when a user of any expertise visits the website. | Duplicated content needs to be removed from the old website. Issue: https://github.com/pytorch/ignite/issues/2325 |
| Users were not satisfied with the clarity in the documentation. | We have simplified the content wherever we could, added prerequisites and links wherever we could. | |
| Need more tutorials on advanced features. | We have 3 intermediate and advanced tutorials. Along with that we have how-to guides and blogs on advanced topics. | Pull Requests: https://github.com/pytorch-ignite/examples/pull/69, https://github.com/pytorch-ignite/examples/pull/49, https://github.com/pytorch-ignite/examples/pull/73 |
| Need more examples on popular vision, text, etc tasks | We have a transformers tutorial trained on IMDb reviews, a machine translation tutorial and a GANs blog on CelebA dataset. | |
| Need more examples of interesting use cases. | How-to guides address different problems that can be solved with Ignite. There are 10 different kinds of guides on the website currently. | |
| Need more FAQs | We converted some of the common questions asked on our Discord and issues to how-to guides. | Pull Requests: https://github.com/pytorch-ignite/examples/pull/71 |
| Need more blogs | We have 3 new blogs on various topics. | |
| Need more illustrations | Our tutorials and blogs utilize custom illustrations wherever needed. One example can be seen in the collective communication tutorial in which we explain different functions through a diagram. | |
| More multi GPU guidance and documentation | Our Distributed Training with CIFAR10 tutorial addresses that. | https://github.com/pytorch-ignite/examples/issues/67 |
| Interactive minimal examples | All our examples are written in notebooks which can quickly run on Google Colab. | |
| Some API docs lack examples | Need to go through the API docs and create PRs where examples are missing | |
| More social media visibility | We have been active on Twitter, Discord and Facebook posting about new announcements, hosting public meetings, etc. |
PyTorch-Ignite presented to you with love by PyTorch community