Why Report On the Future of Open Source?
ㅤ
With headlines that concern the wellbeing of so many, it may seem as if dedication of time and thought to something other than the study of the open source ecosystem's future is needed. However, the ubiquity of open source means that it solves not just business challenges but existential issues for billions of people. One example of millions whose destinies are interwoven with the fate of open source technology is that of refugees. The United Nations High Commissioner for Refugees (UNHCR) agency has documented how vital cell phones are. Reports place connectivity's importance on par or, at times, even above access to food and clean water.¹ Open source is at the beating heart of plotting a passage to safety for vulnerable persons just as much as it is enshrined in the safe transfer of information across corporate, social, and governmental sectors.
ㅤ
One might rightly point out that research into the state of the open source ecosystem has already been achieved through outstanding work done by the TODO Group, the Linux Foundation, and others. This piece is meant to build on that body of knowledge and fill in a perceived insight gap that Major League Hacking is uniquely well-situated to fill. It is our hope that readers of all backgrounds with an interest in open source will find value in its contents. This is a living document that will be updated as insights are garnered from additional research. We hope you might consider getting involved or providing feedback. Please write to [email protected].
Given the vast and deep impact of open source, it makes sense to pay attention to open source “velocity” (OSV): where open source is going and the speed with which it is headed there. To fathom the speed and trajectory of open source, examining GitHub helps one understand the state of open source. GitHub describes itself as the “largest open source community in the world.”²
After the above-mentioned inspection of GitHub as well as an acknowledgment of forces that shape the open source ecosystem, readers use novel data offered by Major League Hacking (MLH) to inform their perspective on OSV. The data are gathered from MLH’s global member community of over five hundred thousand early-career developers.³ The insight offered, though, is not just derived from the early-talent population. Instead, the perspective may be regarded as an emergent property of the efforts made by early-career developers in partnership with developer relations professionals, H.R. veterans, open source maintainers, technical mentors, and distinguished leaders in the Software Engineering industry.
Open Source Software is Foundational to Today's Computing Infrastructure.
How widespread open source is and what results from it have been the study of LF Research (the Linux Foundation’s Research arm), which collaborated with the TODO Group and The New Stack to produce The Fourth Annual Open Source Program Management (OSPO) Survey.
96%
of surveyed large enterprises rely on open source. Of the one hundred ninety-two respondents who indicated that they work for an organization with more than one thousand employees, only eight assert that their products contain no open source components.⁴
80%
of enterprises are said to use open source.
40M
As of August 2020, GitHub was reported to have over 40 million users with more than 37.6 million public repositories (about a quarter of which did not exist in 2018).⁵
An example of consumption of open source without reciprocity is the story of the software library OpenSSL, which began securing communications twenty-four years ago.⁸ According to the technology profiler BuiltWith, OpenSSL is currently used by 2,788,346 live websites, 10,012,631 sites historically, and 1,371,740 websites in the United States.⁹ Despite how thoroughly adopted OpenSSL has been, as of 2019, the project had two full-time employees and a budget of less than one-million USD.¹⁰ A bug in OpenSSL brought international attention to the software library and, in Buzzfeed's April 25, 2014 article titled "The Internet Is Being Protected By Two Guys Named Steve," author Chris Stokel-Walker wrote that OpenSSL had become on par with "critical infrastructure around the world — highways, medical services, and banks." The President of the OpenSSL Software Foundation wrote in 2014 of the funding that came in due to the seismic shift in awareness of the importance of OpenSSL. The historic donation levels rose from two-thousand to nine-thousand dollars per year:¹¹
ㅤ
Thanks to that publicity there has been an outpouring of grassroots support from the OpenSSL user community... all those donations together come to about US$9,000... it is nowhere near enough to properly sustain the manpower levels needed to support such a complex and critical software product. While OpenSSL does “belong to the people” it is neither realistic nor appropriate to expect that a few hundred, or even a few thousand, individuals provide all the financial support. The ones who should be contributing real resources are the commercial companies and governments who use OpenSSL extensively and take it for granted.
ㅤ
ㅤ — Steve Marquess
President & CEO, Veridical Systems
Former President
The OpenSSL Software Foundation
Open Source Sustainability is Struggling As it Never Has Before.
Open Source Velocity allows us to quantify this important insight.
This has never happened before.¹²
After reviewing the data, one may wonder why the sum of closed and open issues does not total to the number of issues reported in the “Issues” column. That is because open issues persist from year to year until they are closed. It is a summative column.
ㅤ
What is being reported in the “Open” column is the number of issues from each of the years examined that still remain open. What is most noteworthy is that the number has never exceeded the number of issues closed in a year until after 2020.
ㅤ
For instance, one may navigate to GitHub and search for open issues that have a created date in a particular year. What one sees here are those issues created in the year 2010 that are still open today.
ㅤ
The number will almost certainly not match the one reported in the table above because issues are constantly being resolved. The rate at which issues are resolved is a key indicator of the sustainability and velocity of open source.
Velocity is the directional speed of an object in motion as an indication of its rate of change in position as observed from a particular frame of reference and as measured by a particular standard of time.
Open Source Velocity (OSV) is the directional speed of the open source ecosystem’s movement notated as the rate at which it may be said to be becoming either more sustainable or less sustainable.
The difference in the steepness of the line graphs for open and closed issues indicates OSV.
With the chart of "Closed and Open," issues illustrated here, one may observe that since 2020, the rate of accumulated open issues has increased over three times as steeply (with a 75% YoY increase) as the rate at which issues are closed (only a 20% YoY increase).
When one subtracts the rate of open issues from the rate of closed issues, one discerns that unresolved issues are accumulating 55% faster than the speed at which issues are being closed. This is a negative, less desirable, less sustainable trend for the open source ecosystem.
This report does not yet factor in:
59% of 204 surveyed maintainers either quit or considered quitting their maintainer role.
Those surveyed were able to select all the reasons that they felt applied. Remarks on the results of the reasons maintainers quit follow.
How does the open source community benefit from monitoring both OSV and maintainer sentiment?
ㅤ
Continued study will show whether there is a correlation between the two metrics. Common sense dictates that maintainers may be less likely to want to walk away if open issues are not overwhelming them.
It is noteworthy that the most cited reason for walking away from projects is work/life changes.
How attributable this is to disruption due to COVID-19 is likely only measurable over a period of years. However, if there has been a breakdown in sustainability and corrective action unlike any that has been witnessed in the past is indeed necessary, it benefits open source's survivability to think ahead in measurable ways.
Will the community see OSV return to pre-2020 levels where closed issues kept pace with issues being opened?
Or, like a black hole's point of no return, has the community plummeted past the event horizon past which no "escape velocity" is achievable?¹³
What can be made of the unprecedented number of unresolved issues and the fact that maintainers are quitting? What do these aspects of open source mean for its future? It comes down to the number of persons handling the issues as well as how that work is being achieved.
That answer is indicated in Clyde Seepersad’s April 2022 article titled “You Can’t Hire Your Way Out of the Cloud Skills Shortage.” Although Mr. Seepersad speaks of the shortfall of cloud computing talent needed, the same holds true for open source talent more broadly. Clyde’s remarks were reported in the software media publication “The New Stack”:
Organizations need to realize that no matter how high of wages they offer, how many acqui-hire deals they engage in, or how heavily they recruit, they are not going to be able to fill all their needs for cloud talent. The only way to face this challenge successfully is to take steps to grow the talent pool itself. There are several actions organizations can take to make progress in this area. For most organizations, it will not be enough to implement one of these actions, rather multiple pieces will be required to get your cloud teams fully staffed. Companies need to adopt an “all of the above” approach to talent acquisition.
— Clyde Seepersad
Senior Vice President & General Manager
Training & Certification, The Linux Foundation
What is clear from Seepersad’s statement is that ecosystems that were once held together via legacy staffing approaches can no longer do so. This is because of the enduring and massive skills gap in the tech sector. For instance, before COVID-19, the American trade organization, The Computing Technology Industry Association (CompTIA), noted a gap of 918,000 tech workers needed in a single quarter.14 And, according to "The Linux Foundation’s 2021 Open Source Jobs Report: 9th Annual Report on Critical Skills, Hiring Trends and Education" the “talent gap that existed before the pandemic has worsened due to an acceleration of cloud-native adoption as remote work has gone mainstream. With talent shortages around the globe, training existing staff has become even more important to meet the needs of migrations to the cloud and leverage open source technologies tied to those migrations."
In summary, yesteryear talent approaches to meet the demands of the present moment are insufficient.
When COVID-19 upended markets, forty thousand MLH community members lost out on internships or their first job offers. Hiring freezes took hold. Burnout ravaged management structures. And it was out of this mayhem that the MLH Fellowship was born. The impact that the Fellowship promises to have on OSV is a positive one that will be detailed in the following section.
A Few Reasons Why Supporting New Developers Helps OSV.
Supporting the next generation of developers is the reason why the MLH Fellowship exists. There are a few reasons why the MLH Fellowship model is successfully beginning to act as a countervailing pressure against the demands dragging down OSV.
The Fellowship is pairing companies that care about Open Source with vetted, promising contributors ready to make a start.
Instead of years, worthwhile contributions from promising developers are being made in weeks.
As an alternative to traditional collegiate internships where a capstone project is ingested by a single company, MLH Fellows render deliverables that serve all those who relying on open source.
Maintainers are happier onboarding and collaborating with new contributors via the MLH Fellowship model.
MLH’s “Season Census” surveyed 14,000 respondents in the Spring of 2022. These charts refer to programmers residing in the US, UK, Canada, and Ireland.
... it is evident that there are some self-limiting beliefs constraining these early-career developers from participating in Open Source.
What is significant about the number of developers in question? How does one weigh that there are as many as fifty software engineers with about a decade of experience saying they either do not know how to get started in open source or that they do not believe themselves to be skilled enough to contribute?
Consider that many popular projects have a single maintainer.
Mr. Eberhardt systematically examined the development toolchain, the set of software libraries acting as dependencies, for the use of Express JS. In his analysis, Colin finds eighty-eight maintainers committing and deploying functionality that is downloaded by public, private, and social sector organizations as well as individuals over forty million times every week.16
Given the outsized importance for hundreds of millions of projects that even a single maintainer could have, the community benefits from thoughtfully integrating those with even only a little experience to do suitable tasks. Coders may still find ways to contribute to necessary open source assignments given the right guard rails.
Author of “Land the Tech Job You Love,” Andy Lester, wrote in 2012 for Smart Bear.15
In a traditional internship, an assignment generates a deliverable that belongs to the company that is hosting the intern. To help with OSV, MLH Fellows are, instead, making public contributions to open source software that nearly all companies depend on. By contributing publicly to projects that undergird the efforts of numerous endeavors rather than only to one organization, MLH Fellows model a path toward a sustainable vision of an open source ecosystem.
The momentum behind the MLH Fellowship helps OSV.
Although there is considerable interest in the Fellowship,18 it is insufficient, though, to simply add contributors to the Open Source ecosystem. Maintainer happiness is also essential.
ㅤ
ㅤ
Because MLH Fellows receive a stipend, have passed their technical interview, and receive guidance from a technical mentor, maintainers are able to receive much-needed reinforcement as a direct result of MLH Fellows’ involvement in their projects. Of 42 maintainers surveyed, despite MLH Fellows being early-career talent, nearly half of maintainers said that they spend just as much or less time onboarding MLH Fellows as they do with other new contributors.
ㅤ
Also:
Conclusion.
Considering what has been presented here, here are items worth highlighting.
The ubiquity of open source does not indicate its sustainability. Much, in fact, points toward open source's widespread adoption as presenting seemingly untenable resourcing challenges. However, those challenges can begin to be addressed by tuning into core metrics like OSV. These should signal the direction that must be tacked to in order to perserve open source.
Powerful cross-functional coalitions of software engineers, diverse new talent, developer relations, hiring management, and those who care about open source's survival are coming together. More must be done. In the wake of the mayhem wrought by COVID-19 in the Spring of 2020, partners across the ecosystem developed a way of working. Continued study is finding that the benefits are scaling and continued research is called for to see that this continues.
Metrics are not enough. They are helpful for anticipating the changes that are coming around the bend. However, they are not a substitute for the moral imagination needed to recognize the true importance of the open source ecosystem. Programatically encouraging open source development is a must as is diversifying the set of contributors who support the applications on which we rely. The time to invest in the next generation of developers is now.
This report has been brought to you by Major League Hacking (MLH), a global community of over five-hundred thousand rising technologists. Email us with your criticisms, praise, suggestions, thoughts, concerns, and ideas at [email protected].