Open Source Velocity (OSV):Directions & Trends in the Future of Open Source.Mapping a Path to a Sustainable Open Source Ecosystem.

Why Report On the Future of Open Source?

     With headlines that concern the wellbeing of so many, it may seem as if dedication of time and thought to something other than the study of the open source ecosystem's future is needed. However, the ubiquity of open source means that it solves not just business challenges but existential issues for billions of people. One example of millions whose destinies are interwoven with the fate of open source technology is that of refugees. The United Nations High Commissioner for Refugees (UNHCR) agency has documented how vital cell phones are. Reports place connectivity's importance on par or, at times, even above access to food and clean water.¹ Open source is at the beating heart of plotting a passage to safety for vulnerable persons just as much as it is enshrined in the safe transfer of information across corporate, social, and governmental sectors.

     One might rightly point out that research into the state of the open source ecosystem has already been achieved through outstanding work done by the TODO Group, the Linux Foundation, and others. This piece is meant to build on that body of knowledge and fill in a perceived insight gap that Major League Hacking is uniquely well-situated to fill. It is our hope that readers of all backgrounds with an interest in open source will find value in its contents. This is a living document that will be updated as insights are garnered from additional research. We hope you might consider getting involved or providing feedback. Please write to [email protected].

open-source-velocity-osv-mapping-path-to-sustainability
Insight MLH Offers As A Nexus of Several Communities

What Readers Will Learn.

     Given the vast and deep impact of open source, it makes sense to pay attention to open source “velocity” (OSV): where open source is going and the speed with which it is headed there. To fathom the speed and trajectory of open source, examining GitHub helps one understand the state of open source. GitHub describes itself as the “largest open source community in the world.”²

     After the above-mentioned inspection of GitHub as well as an acknowledgment of forces that shape the open source ecosystem, readers use novel data offered by Major League Hacking (MLH) to inform their perspective on OSV. The data are gathered from MLH’s global member community of over five hundred thousand early-career developers.³ The insight offered, though, is not just derived from the early-talent population. Instead, the perspective may be regarded as an emergent property of the efforts made by early-career developers in partnership with developer relations professionals, H.R. veterans, open source maintainers, technical mentors, and distinguished leaders in the Software Engineering industry.

Field of Red Dots

Open Source Software is Foundational to Today's Computing Infrastructure.

     How widespread open source is and what results from it have been the study of LF Research (the Linux Foundation’s Research arm), which collaborated with the TODO Group and The New Stack to produce The Fourth Annual Open Source Program Management (OSPO) Survey.

96%

of surveyed large enterprises rely on open source. Of the one hundred ninety-two respondents who indicated that they work for an organization with more than one thousand employees, only eight assert that their products contain no open source components.⁴

80%

of enterprises are said to use open source.

40M

As of August 2020, GitHub was reported to have over 40 million users with more than 37.6 million public repositories (about a quarter of which did not exist in 2018).⁵

Open source is woven into operating systems (Linux), browsers (Chromium, Firefox), the Signal Protocol library used by WhatsApp, databases (PostgreSQL), machine learning frameworks (TensorFlow), and cloud computing services (Amazon Web Services alone is running twelve hundred open source projects).⁶

Firefox
React
AWS Amplify
Linux
Chromium
postgresql
Babel
Java

Ubiquity Does Not Mean Sustainability.

     An example of consumption of open source without reciprocity is the story of the software library OpenSSL, which began securing communications twenty-four years ago. According to the technology profiler BuiltWith, OpenSSL is currently used by 2,788,346 live websites, 10,012,631 sites historically, and 1,371,740 websites in the United States. Despite how thoroughly adopted OpenSSL has been, as of 2019, the project had two full-time employees and a budget of less than one-million USD.¹⁰ A bug in OpenSSL brought international attention to the software library and, in Buzzfeed's April 25, 2014 article titled "The Internet Is Being Protected By Two Guys Named Steve," author Chris Stokel-Walker wrote that OpenSSL had become on par with "critical infrastructure around the world — highways, medical services, and banks." The President of the OpenSSL Software Foundation wrote in 2014 of the funding that came in due to the seismic shift in awareness of the importance of OpenSSL. The historic donation levels rose from two-thousand to nine-thousand dollars per year:¹¹

Thanks to that publicity there has been an outpouring of grassroots support from the OpenSSL user community... all those donations together come to about US$9,000... it is nowhere near enough to properly sustain the manpower levels needed to support such a complex and critical software product. While OpenSSL does “belong to the people” it is neither realistic nor appropriate to expect that a few hundred, or even a few thousand, individuals provide all the financial support. The ones who should be contributing real resources are the commercial companies and governments who use OpenSSL extensively and take it for granted.

        — Steve Marquess
             President & CEO, Veridical Systems
             Former President
             The OpenSSL Software Foundation

Open Source Sustainability is Struggling As it Never Has Before.

Open Source Velocity allows us to quantify this important insight.

null

After 2020, the number of unresolved issues exceeded the number closed in the year.

This has never happened before.¹²

GitHub Data

About This Data...Remember: Open Issues Accumulate Until They're Closed.

Summative.

     After reviewing the data, one may wonder why the sum of closed and open issues does not total to the number of issues reported in the “Issues” column. That is because open issues persist from year to year until they are closed. It is a summative column.

Unprecedented.

     What is being reported in the “Open” column is the number of issues from each of the years examined that still remain open. What is most noteworthy is that the number has never exceeded the number of issues closed in a year until after 2020.

For instance, one may navigate to GitHub and search for open issues that have a created date in a particular year. What one sees here are those issues created in the year 2010 that are still open today.

Dynamic.

     The number will almost certainly not match the one reported in the table above because issues are constantly being resolved. The rate at which issues are resolved is a key indicator of the sustainability and velocity of open source.

The steepness of closed issues is 20% year over year as opposed to the 75% increase seen with open issues

Defining & Measuring OSVOpen Source Velocity (OSV)

What is Open Source Velocity?

     Velocity is the directional speed of an object in motion as an indication of its rate of change in position as observed from a particular frame of reference and as measured by a particular standard of time.

     Open Source Velocity (OSV) is the directional speed of the open source ecosystem’s movement notated as the rate at which it may be said to be becoming either more sustainable or less sustainable.

Measuring OSV.

     The difference in the steepness of the line graphs for open and closed issues indicates OSV.

     With the chart of "Closed and Open," issues illustrated here, one may observe that since 2020, the rate of accumulated open issues has increased over three times as steeply (with a 75% YoY increase) as the rate at which issues are closed (only a 20% YoY increase).

     When one subtracts the rate of open issues from the rate of closed issues, one discerns that unresolved issues are accumulating 55% faster than the speed at which issues are being closed. This is a negative, less desirable, less sustainable trend for the open source ecosystem.

GitHub Data

If issues are indicative of the state of open source, additional investigation into the status of those issues beyond the open and closed category is relevant.

This report does not yet factor in:

  • + How much time it takes until an issue is resolved for the first time.

  • + The inquiry also does not make distinction between projects of different license types.

  • + A future examination should also factor in how many issues are re-opened as well as the disposition of closed issues (e.g. "won't do, duplicate, etc.").

  • + Finally, at a later date, an examination of the dependency graph, the way in which various projects' functionalities rest upon one another, should also be conducted.
119 Quit or Considered Quitting According to Tidelift in 2021

Data about maintainers underscores strain on the open source ecosystem.

59% of 204 surveyed maintainers either quit or considered quitting their maintainer role.

    Why Maintainers Consider Quitting

    This begs the question...Why are maintainers quitting or thinking of quitting?

    Those surveyed were able to select all the reasons that they felt applied. Remarks on the results of the reasons maintainers quit follow.

    How does the open source community benefit from monitoring both OSV and maintainer sentiment?

    question mark

    Measurement Enables Change.

         Continued study will show whether there is a correlation between the two metrics. Common sense dictates that maintainers may be less likely to want to walk away if open issues are not overwhelming them.

         It is noteworthy that the most cited reason for walking away from projects is work/life changes.

         How attributable this is to disruption due to COVID-19 is likely only measurable over a period of years. However, if there has been a breakdown in sustainability and corrective action unlike any that has been witnessed in the past is indeed necessary, it benefits open source's survivability to think ahead in measurable ways.

         Will the community see OSV return to pre-2020 levels where closed issues kept pace with issues being opened?

         Or, like a black hole's point of no return, has the community plummeted past the event horizon past which no "escape velocity" is achievable?¹³

    Industry Talent Quotes

    What About Investing in Staffing?

          What can be made of the unprecedented number of unresolved issues and the fact that maintainers are quitting? What do these aspects of open source mean for its future? It comes down to the number of persons handling the issues as well as how that work is being achieved.

         That answer is indicated in Clyde Seepersad’s April 2022 article titled “You Can’t Hire Your Way Out of the Cloud Skills Shortage.” Although Mr. Seepersad speaks of the shortfall of cloud computing talent needed, the same holds true for open source talent more broadly. Clyde’s remarks were reported in the software media publication “The New Stack”:

    Organizations need to realize that no matter how high of wages they offer, how many acqui-hire deals they engage in, or how heavily they recruit, they are not going to be able to fill all their needs for cloud talent. The only way to face this challenge successfully is to take steps to grow the talent pool itself. There are several actions organizations can take to make progress in this area. For most organizations, it will not be enough to implement one of these actions, rather multiple pieces will be required to get your cloud teams fully staffed. Companies need to adopt an “all of the above” approach to talent acquisition.


             — Clyde Seepersad
                 Senior Vice President & General Manager
                 Training & Certification, The Linux Foundation

         What is clear from Seepersad’s statement is that ecosystems that were once held together via legacy staffing approaches can no longer do so. This is because of the enduring and massive skills gap in the tech sector. For instance, before COVID-19, the American trade organization, The Computing Technology Industry Association (CompTIA), noted a gap of 918,000 tech workers needed in a single quarter.14 And, according to "The Linux Foundation’s 2021 Open Source Jobs Report: 9th Annual Report on Critical Skills, Hiring Trends and Education" the “talent gap that existed before the pandemic has worsened due to an acceleration of cloud-native adoption as remote work has gone mainstream. With talent shortages around the globe, training existing staff has become even more important to meet the needs of migrations to the cloud and leverage open source technologies tied to those migrations."

         In summary, yesteryear talent approaches to meet the demands of the present moment are insufficient.

         When COVID-19 upended markets, forty thousand MLH community members lost out on internships or their first job offers. Hiring freezes took hold. Burnout ravaged management structures. And it was out of this mayhem that the MLH Fellowship was born. The impact that the Fellowship promises to have on OSV is a positive one that will be detailed in the following section.

    A Few Reasons Why Supporting New Developers Helps OSV.

    Supporting the next generation of developers is the reason why the MLH Fellowship exists. There are a few reasons why the MLH Fellowship model is successfully beginning to act as a countervailing pressure against the demands dragging down OSV.

    Pairing Talent with Companies

    The Fellowship is pairing companies that care about Open Source with vetted, promising contributors ready to make a start.

    Weeks. Not Years.

    Instead of years, worthwhile contributions from promising developers are being made in weeks.

    Deliverables That Serve All.

    As an alternative to traditional collegiate internships where a capstone project is ingested by a single company, MLH Fellows render deliverables that serve all those who relying on open source.

    Maintainers are happier.

    Maintainers are happier onboarding and collaborating with new contributors via the MLH Fellowship model.

    Ever Contributed to Open Source

    Contribute in Just Weeks. Not Years.

    MLH’s “Season Census” surveyed 14,000 respondents in the Spring of 2022. These charts refer to programmers residing in the US, UK, Canada, and Ireland.

    I am not skilled enough

    There are ~4,200 respondents who are not contributing to Open Source but plan to do so in the future.Among approximately half of those persons...

    ... it is evident that there are some self-limiting beliefs constraining these early-career developers from participating in Open Source.

    Don't know how to start

    Remedying this confidence gap can unlock massive amounts of developer talent.

    • 1.  More than half of the respondents who do not contribute to open source, over twelve hundred, are coders with more than three years of experience. And yet, they do not believe they have the skills necessary to contribute to open source. Or they say they do not know how to get started.
    • 2.  More than five hundred programmers not contributing to open source indicate they possess no less than five years of experience.
    • 3.  Around fifty-five have practiced for about a decade.

    What is significant about the number of developers in question? How does one weigh that there are as many as fifty software engineers with about a decade of experience saying they either do not know how to get started in open source or that they do not believe themselves to be skilled enough to contribute?

    Consider that many popular projects have a single maintainer.

    Colin Eberhardt, CTO, Scott Logic; FINOS Leeds, 2021.

    Consider, too a 2021 presentation by Colin Eberhardt to the Linux Foundation's FinTech Open Source Foundation (FINOS) in Leeds.
    The dependencies of the Express JS toolchain are controlled by just eighty-eight developers.

    Mr. Eberhardt systematically examined the development toolchain, the set of software libraries acting as dependencies, for the use of Express JS. In his analysis, Colin finds eighty-eight maintainers committing and deploying functionality that is downloaded by public, private, and social sector organizations as well as individuals over forty million times every week.16

    Given the outsized importance for hundreds of millions of projects that even a single maintainer could have, the community benefits from thoughtfully integrating those with even only a little experience to do suitable tasks. Coders may still find ways to contribute to necessary open source assignments given the right guard rails.

    MLH Fellows and Partner Logos

    Deliverables That Serve All.

    In a traditional internship, an assignment generates a deliverable that belongs to the company that is hosting the intern. To help with OSV, MLH Fellows are, instead, making public contributions to open source software that nearly all companies depend on. supporting-projects-like By contributing publicly to projects that undergird the efforts of numerous endeavors rather than only to one organization, MLH Fellows model a path toward a sustainable vision of an open source ecosystem.

    GitHub View MLH Fellowship First Year

    The number of applications increased from twenty thousand to thirty thousand in just one year.

    The momentum behind the MLH Fellowship helps OSV.

    Maintainer Happiness Improves.

    Although there is considerable interest in the Fellowship,18 it is insufficient, though, to simply add contributors to the Open Source ecosystem. Maintainer happiness is also essential.

    • · 80% of maintainers say that they need help finding and recruiting new contributors.
    • · 90% say they need help improving the new contributor experience.17

    Because MLH Fellows receive a stipend, have passed their technical interview, and receive guidance from a technical mentor, maintainers are able to receive much-needed reinforcement as a direct result of MLH Fellows’ involvement in their projects. Of 42 maintainers surveyed, despite MLH Fellows being early-career talent, nearly half of maintainers said that they spend just as much or less time onboarding MLH Fellows as they do with other new contributors.

    Also:

    • · 68% indicate the Fellowship made them more positive about being an open source maintainer.
    • · 60% expect MLH Fellows to continue to contribute after the Fellow wraps their time on the Fellowship.

    Conclusion.

    Considering what has been presented here, here are items worth highlighting.

    Ubiquity ≠ Sustainability

    The ubiquity of open source does not indicate its sustainability. Much, in fact, points toward open source's widespread adoption as presenting seemingly untenable resourcing challenges. However, those challenges can begin to be addressed by tuning into core metrics like OSV. These should signal the direction that must be tacked to in order to perserve open source.

    Cross-Functional Coalitions Must Come Together

    Powerful cross-functional coalitions of software engineers, diverse new talent, developer relations, hiring management, and those who care about open source's survival are coming together. More must be done. In the wake of the mayhem wrought by COVID-19 in the Spring of 2020, partners across the ecosystem developed a way of working. Continued study is finding that the benefits are scaling and continued research is called for to see that this continues.

    Systemic Change Takes Effort & Time

    Metrics are not enough. They are helpful for anticipating the changes that are coming around the bend. However, they are not a substitute for the moral imagination needed to recognize the true importance of the open source ecosystem. Programatically encouraging open source development is a must as is diversifying the set of contributors who support the applications on which we rely. The time to invest in the next generation of developers is now.

    Get Involved

    This report has been brought to you by Major League Hacking (MLH), a global community of over five-hundred thousand rising technologists. Email us with your criticisms, praise, suggestions, thoughts, concerns, and ideas at [email protected]

    null

    References

           
    1. 1. Vernon, A., Deriche, K. and Eisenhauer, S., 2016. Connecting Refugees: How Internet and Mobile Connectivity can Improve Refugee Well-Being and Transform Humanitarian Action. United Nations High Commissioner for Refugees. [Accessed 8 June 2022].
    2. 2. Njisse, Jeff. "Mining GitHub to Identify Open-Source Software Health in Blockchain Projects." Rangahau Aranga: AUT Graduate Review 1.1 (2022).
    3. 3. Wikipedia contributors. "Major League Hacking." Wikipedia, The Free Encyclopedia. Wikipedia, The Free Encyclopedia, 23 May. 2022. [Wikipedia] 23 May. 2022.
    4. 4. TODO Group, 2021. Open Source Program Office (OSPO) 2021 Survey. [CSV] Available at: GitHub [Accessed 27 May 2022].
    5. 5. Wu, Qiushi, and Kangjie Lu. "On the feasibility of stealthily introducing vulnerabilities in open-source software via hypocrite commits." Proc. Oakland, page to appear (2021).
    6. 6. MacManus, R., 2022. The Open Source Strategy of Amazon Web Services. [online] Thenewstack.io. The New Stack [Accessed 21 December 2020].
    7. 7. Littauer, R., Nickolls, B. and Eberhardt, C., 2022. Colin Eberhardt of Scott Logic on Software Sustainability. [podcast] Sustain. Available at: SustainOSS [Accessed 3 June 2022].
    8. 8. Laurie, Ben (January 6, 1999). "ANNOUNCE: OpenSSL (Take 2)". ssl-users Mailing list. [Accessed 6 June 2022].
    9. 9. Trends.builtwith.com. 2022. OpenSSL Usage Statistics. [online] Available at: BuiltWith Trends [Accessed 6 June 2022].
    10. 10. "New Committers". OpenSSL Software Foundation. May 20, 2019. Retrieved November 3, 2019.
    11. 11. Marquees, S., 2014. Of Money, Responsibility, and Pride. Veridicalsystems.com. [Accessed 6 June 2022].
    12. 12. Kagan, R., 2022. GitHub Issues Created by Year [Accessed 16 May 2022].
    13. 13. National Aeronautics and Space Administration (NASA) Goddard Space Flight Center, 2016. Imagine the Universe: Black Holes [Accessed 8 June 2022]
    14. 14. Loten, A., 2019. America’s Got Talent, Just Not Enough in IT. The Wall Street Journal. [Accessed 17 May 2022].
    15. 15. Lester, A., 2012. 14 Ways to Contribute to Open Source without Being a Programming Genius or a Rock Star. [online] SmartBear.com. [Accessed 26 May 2022].
    16. 16. Colin Eberhardt of Scott Logic on Open Source Software Sustainability & Corporate Social Responsibility. [webcast] Open Source Leeds. [Accessed 6 June 2022].
    17. 17. Logan, M., 2021. The 2021 Tidelift open source maintainer survey. [Accessed 16 May 2022].
    18. 18. Gennarelli, V., 2020. Welcome to the inaugural class of MLH Fellows | The GitHub Blog. [online] The GitHub Blog. [Accessed 27 May 2022].