Skip to content

Google Summer of Code 2018

Jump to bottom
Neelabh Gupta edited this page Jan 23, 2018 · 24 revisions

GSoC 2018

At Twitter, we love Open Source, working with students and Google Summer of Code (GSOC)! What is GSOC? Every year, Google invites students to come up with interesting problems for their favorite open-source projects and work on them over the summer. Participants get support from the community, plus a mentor who makes sure you don't get lost and that you meet your goals. Aside from the satisfaction of solving challenging problems and contributing to the open source community, students get paid and get some sweet swag for their work! In our opinion, this is a great opportunity to get involved with open source, improve your skills and help out the community!

Information for Students

These ideas were contributed by our developers and our community, as a starting point. If you wish to submit a proposal based on these ideas, you may wish to contact the developers and find out more about the particular suggestion you're looking at.

Being accepted as a Google Summer of Code student is quite competitive. Accepted students typically have thoroughly researched the technologies of their proposed project and have been in frequent contact with potential mentors. Simply copying and pasting an idea here will not work. On the other hand, creating a completely new idea without first consulting potential mentors is unlikely to work out.

If there is no specific contact given you can ask questions via the twitter-gsoc mailing list.

Project Ideas

Improve Finatra HTTP Client functionality and API

  • Project Category: Core Development
  • Project License: Apache 2.0
  • Detailed description of the project:
    • Finatra is Twitter's Scala framework for easily building microservices. It it primarily an MVC framework which supports both HTTP and Thrift protocols with a focus on testability. It used in hundreds of services at Twitter. Your team's tasks will be to update the Finatra HTTP Client library to the Finagle 6 APIs via Finagle's MethodBuilder for clients and provide a more robust API for users trying to connect to HTTP services.
    • You will be adding features including but not limited to:
      • Support flexible retry and response classification, proxy configuration.
      • Add support for the HttpClientModule to bind multiple http clients to the object graph (see Finatra user guide section on binding annotations).
      • Add more testing.
    • The goal is to accomplish this in a backwards-compatible way if possible, but if necessary, we can deprecate current approach for a a new parallel API and implementation.
  • Expected outcomes for Student:
    • A chance to contribute and work on a foundational piece of the Finatra framework which is a central piece of Twitter's backend infrastructure and microservices architecture. A well-functioning http client (à la Apache's HTTPClient) in Scala and built on Finagle is missing. The Finatra and Finagle communities would be very appreciative of work in this area.
  • Deliverables:
    • Design document (based on provided Twitter Template)
    • Port to new Finagle 6 API
    • Use new MethodBuilder API/Classes
    • Implement new Client functionality from Design Doc
    • Specifying Retries for requests
    • Specify Response Classification (stop on 500? Look at errors on a 200?)
    • Proxy Configuration (exists in Finagle, doesn't exist here)
  • Skills required: HTTP fundamentals, data structures
  • Skills preferred: git, GitHub, Scala, API design, dependency injection experience (ideally with Guice), experience with RPC
  • Mentors:
  • Difficulty Rating: Intermediate
  • Data Sensitivity: None.

Improve Finatra HTTP routing

  • Project Category: Core development
  • Project License: Apache 2.0
  • Detailed description of the project:
    • Finatra is Twitter's Scala framework for easily building microservices. It it primarily an MVC framework which supports both HTTP and Thrift protocols with a focus on testability. It used in hundreds of services at Twitter.
    • Your team's tasks will be to update the way that Finatra HTTP routing works in a backward-compatible way to make it more efficient with an eye to solving several problems:
      • It is not currently possible to correctly return a 405 "Method Not Allowed" from the framework and this is directly due to how routing works.
      • Improve speed and efficiency of route matching.
  • Expected outcomes for Student:
    • This is a chance for someone to impact all HTTP services written with Finatra, including hundreds of services inside Twitter, and multiple other companies. If the speed and efficiency of http route matching can be improved in Finatra, lots of services both inside and outside of Twitter would benefit.
    • What you'll learn:
      • In depth experience with benchmarking, and performance testing.
      • Deep understanding of HTTP protocol and systems.
      • Advanced Scala and data structures exposure.
  • Deliverables:
    • Design document (based on provided Twitter Template)
    • It is not currently possible to correctly return a 405 "Method Not Allowed" from the framework and this is directly due to how routing works.
      • Determine the 'verb' and the 'uri' you're applying it to.
      • Support a given verb for a given route is current paradigm.
    • Improve speed and efficiency of route matching.
      • Create and update existing benchmarks.
      • Deliver regular reports and visualizations on benchmarks.
      • Experiment with new data-structures (currently recursive looping).
      • Repeat.
  • Skills required: HTTP fundamentals, data structures
  • Skills preferred: git, GitHub, Scala, API design, dependency injection experience (ideally with Guice), benchmarking/performance testing, data visualization
  • Mentors:
  • Difficulty Rating: Advanced
  • Data Sensitivity: None

rpc-perf Advanced Benchmarking Strategies

  • Category: Core Development
  • License: Apache 2.0
  • Detailed description of the project:
    • rpc-perf is a high-performance benchmarking tool for measuring the performance of services like Twemcache, Memcache, and Redis. You will develop core features of rpc-perf to allow for advanced workload generation strategies. Desired features include sharding of requests using consistent hashing, sharding-aware key generation to create hot-spots, generating workload from log-replay, latency-based rate-limiting, and backoff if success-rate drops. Implement advanced features which allow for dynamically probing and characterizing performance across a variety of workloads.
  • Expected outcomes for students:
    • This tool is used internally and is also linked off of https://redis.io/topics/benchmarks. Any enhancement and new advanced functionality will help our engineers in understanding performance of caching services; additionally, these features are likely to appeal to the broader community. You will work with high-performance load-generating and networking code to add features that will make it easier to measure and characterize the performance of distributed services. You will learn about how performance of distributed systems are measured and evaluated. You will get hands-on experience with async networking and event loops. You will be exposed to network client design and connection management.
  • Deliverables:
    • Requirements Docs (provided by Twitter Mentor)
    • Design docs / RFCs for complex features (provided by Twitter Mentor)
    • Open GitHub issues for all core development items (provided by Twitter Mentor)
    • Each new complex feature added as separate milestones.
    • Student to work from open issues on GitHub and contribute patches which extend the functionality of rpc-perf. These issues may be approached incrementally, with more complex features building upon prior work. At each step, the pull requests will be reviewed and refined until merged
  • Skills required: Languages: Rust, OS: Linux, Systems Programming, Networking
  • Skills preferred: Linux performance analysis (perf), Network protocol analysis (tcpdump, wireshark), Experience writing client libraries (connection pooling/management, codecs)
  • Mentors:
  • Difficulty Rating: Intermediate/Advanced
  • Data Sensitivity: None. Existing Twitter OSS project.

Pants Build Website Redesign

  • Category: Infrastructure/Automation
  • License: Apache 2.0
  • Detailed description of the project:
    • Pants is a collaborative open-source project, built and used by Twitter, Foursquare, Square, Medium and other companies. Your project will be to design, build and revamp the pantsbuild.org documentation website.
  • Expected outcomes for Student:
    • Huge impact within the Open Source Community - Pantsbuild already has users but making the documentation site clearer and more accessible will open up the platform for more eager adoption and even to better support the existing users.
    • Why it's interesting:
      • Work with Twitter Build Team members directly!
      • Pants is an upstream project used by other projects and organizations too (Square, Kayak, Foursquare, Compass, and others).
    • What you'll learn:
      • Open Source Build Systems
      • Advanced Python Techniques
      • Remote Collaboration
    • Impact for Twitter:
      • Pants is what we use to build Twitter -- thousands of devs depend on it.
      • Pants is used by other projects and organizations too (Square, Kayak, Foursquare, Compass, and others).
  • Deliverables:
  • Skills required: HTML, CSS, Git, Technical Writing
  • Skills preferred: JavaScript, Python, Graphic Design tools like photoshop, UI/UX Experience, Github/Github Pages
  • Mentors:
  • Difficulty Rating: Beginner/Intermediate
  • Data Sensitivity: No, it's open source.

Student Proposals

Proposals must follow GSoC Guidelines (see: https://google.github.io/gsocguides/student/writing-a-proposal)

Proposals must be submitted via http://summerofcode.withgoogle.com.

Proposals must be drafted in plain-text, but final submission format is required to be a .pdf.

  • Name
  • Twitter Handle
  • GitHub ID
  • Email Address
  • Proposal Title
  • Synopsis
  • Benefits to Community
  • Deliverables (in a week-by-week format)
  • Related Work
  • Biographical Information (brief, and focused on Open Source experience)

Important Guidance

Communicate early and often

If the first time we have heard from you is when you submit your final application, it will be difficult to know how best you fit into the community, or what it's like to work with you as a contributor. Clone the upstream repo and experiment with the code base. Join the mailing list and introduce yourself early. Share your proposal as soon as you've got all the elements, and ask for feedback before you submit.

Develop your proposal with revision control

Whether it's a Google Doc, or a gist or repo on GitHub, the ability to share and annotate changes, and view/revert history will help us give quality feedback.

Proposals should be in the range of ~1000 words

Be sure to include all of the required sections above, and not stray too far above or below the wordcount. Again, Actions Speak Louder Than Words, so the manner in which you iterated and collaborated to create the content is just as important as the content itself.

Good luck!

Accepted Projects

For 2018, @TwitterOSS accepted X students to work on Y different open source projects:

Follow us at @TwitterOSS

Google Summer of Code 2018 Projects

Clone this wiki locally