Perhaps surprisingly: I don’t actually give the tiniest shit whether he was ignorant, malicious or a “good person” and: you shouldn’t either.
Imagine you have two radically different coworkers: Bob and Alice.
The problem is, though: people like Bob seem to never quite get fired and, too often, people like Alice are labelled as “difficult”, “unfriendly” or, most disgustingly, a “poor culture fit”. We cut Bobs far too much Slack and Alices get insufficient credit.
A brief tangent: I used to be somewhat religious. One thing that struck me in those days was the hostility towards post-modern claims of the absence of “universal truth”. It always seemed a funny thing to get annoyed about; particularly when those who would get annoyed could not confidently state what the “universal truth” was. I could never see much of a difference between:
It feels like there’s a similar thing going on with judging people’s intentions. Despite my best attempts, I cannot draw a meaningful distinction between:
As a result: I no longer care about intentions.
I’ve decided I love working with Alices and have developed an aversion to Bobs. If their actions are incompetent, hurtful and disruptive: that’s what I care about. I don’t care about whether they are trying their best or not. The people they hurt or help certainly don’t. Whether they’re in my personal life, open-source community or at work: I no longer have room for the repeatedly hurtful, even if their hearts are in the right place. If they are over 25 years old, chances are they should have figured out by now how people are hurt and how they can be helped.
Intentions be damned: it’s about doing good work and helping people.
]]>My favourite procrastination time sinks have been, in chronological order:
Most of these have been conquered by tools providing child-like parental limits rather than abstinence None of these could have been improved through willpower alone as I don’t seem to have any.
This was the easiest to solve. In increasing order of effectiveness I:
I still play a fair few games, though. It’s a great source of relaxation for me that I can do in short bursts, at home, for a relative pittance.
I wrote a post about this at the time but, TL;DR, I spent way too much time fucking around with my setup. Repeatedly refucking and defucking my Linux system was not a good use of my time. I still grew some neckbeard skills along the way, though, that proved very useful later when doing Linux server work and maintaining Homebrew.
I am effectively “abstinent” on desktop Linux, though. My innate urge to tinker makes it a terrible idea to do otherwise.
I wrote previously about nuking my old content from Facebook and Twitter. I still use some social networks to a small extent but no longer allow social networks are allowed as apps or logged-in websites on my phone. Ultimately, I don’t find Mastodon (my primary social network nowadays) to be too much of a mind virus on my work computer. I learn useful things and have positive interactions with people that I’d miss if I went full abstinence.
I’m highly privileged enough to be mostly unaffected by most political changes and most news stories. I was also “lucky” enough to lose touch with good, real-life friends in early social networking days over stupid political arguments. As a result, I’ll still occasionally talk and think about politics and news but I try to only argue about it in meatspace, ideally: over a pint. I use an iOS app, ScreenZen, which uses Apple’s ScreenTime API (more on that later) to make it as unpleasant as possible to read it on my phone without fully blocking it.
These are a tricky pair because they are useful to me sometimes but I get massively diminishing returns the more time I spent spiralling into their bullshit.
Hacker News has a nice server-side “procrastination mode” (noprocrast
in your user settings): this lets you limit how long I can spend pissing around before you are blocked from doing so.
Reddit is like eating crisps for me: mostly bad for me but sometimes I need to trust my more-intense-than-usual need for it and indulge myself. As a result, I have heavily limited it by:
This all might sound like a lot and you may think I have great self-control as a result. Wrong. I have to put all these policies and restrictions in place because I have zero self-control.
I am glad for these changes overall, though. The result has been a better mood, being more productive at work and being more present with family.
You’ll still see me glued to my phone sometimes, though, but I’m probably just reading a new science fiction novel; a hobby nearly driven to extinction by the omnipresence of technology.
Good luck and happy procrastination-battling!
]]>If you’re still on the fence about seeing a therapist at all, you might find these anonymous therapy podcasts to be informative and interesting:
If you’re still on the fence: try it and see if it helps.
Good luck!
]]>I was introduced to this book in a Human-Computer Interaction course at university but most of it barely mentions computers at all. It radically changed the way I thought about design of all everything, including all the software I have written since, to aim to be as intuitive and natural as possible. Tip: read this on paper or on an iPad rather than a iPhone or Kindle as the pictures as essential.
I was lucky enough to read an early draft of this book and it’s simply the best analysis of the open-source software ecosystem around today. The writing style is friendly and not formal despite the academic-level research that went into this book. It made me think differently about the open source projects I maintain and how I interact and chose those I use.
This is the oldest book on my list and is the most underrated. It describes, with serious rigour and detail, how to run more effective software projects and teams. Most of this advice has been ignored my most of the industry for most of the time but it’s a big part of the reason I’ve worked from home for 14 years and am as productive as I am today.
This is from an older generation of open source development, pre-GitHub, but much of the advice here is still incredibly relevant and astute today, helping provide advice on the interpersonal as well as technical sides to open source development. Many of the underlying principals from this book are what form my grounding as an open source maintainer for the last 15 years.
Another book that predates GitHub but provides a lot of actionable advice today that’s been mostly ignored across much of our industry. Hard problems like “why isn’t my software project reliable?” are tackled head on and addressed here. It was an early nudge for me to automate as much as possible in software projects and not ignore difficult problems with project organisation in favour of “fun” technical tasks.
Originally posted on Shepherd.com as “The best books for becoming a great open source software engineer”.
]]>log4j
vulnerability about whether or not open source is broken or sustainable, what we can do to improve the sustainability of the open source ecosystem moving forwards, and the entitlement of users and companies in expecting maintainers to fix their problems.
As the project leader of Homebrew, a macOS and Linux package manager with millions of users, I have experienced, and continue to experience, a lot of entitled behaviour from contributors and users of Homebrew. Ironically, this is often worse coming from employed developers’ large tech companies with fantastic profit margins. Similarly, being a staff engineer based in Scotland at GitHub and working in a very different time zone to my peers, I have learned that setting consistent, clear and firm boundaries with people in my open source and professional work results in a better experience for (almost) everyone.
Let’s start with a few definitions of terms I’ll use in this article so we’re all on the same page:
To understand entitlement in open source, let’s use a fictional case study based on my experiences.
Bob works for TechCorp and discovered a few years ago that using a tool installed from Homebrew results in a 90% speedup on an otherwise boring, manual task he has to perform regularly. This tool is increasingly integrated into tooling, documentation and process at TechCorp and everyone is happy, particularly Bob. Bob receives a good performance review for improving the process at TechCorp.
Fast-forward a few years, one of Bob’s coworkers runs the tool that Bob built and it doesn’t work as expected, throwing a weird error no one has seen before. They go to Bob and ask what the problem is. Bob is in the middle of another project with a tight deadline so doesn’t really have time to help but feels some responsibility and jumps in and checks it out.
Bob discovers that a change in Homebrew has meant that the tooling will no longer work the way it used to. As he digs in, he realises this was announced by Homebrew a few months ago but he never saw the messages.
Frustrated and stressed, Bob opens an issue on Homebrew where he makes clear how disruptive this change has been to him and his organisation, who are heavy users of Homebrew, and that it needs a fix ASAP so they can deliver what they have promised to customers. His open issue awaits a response.
Reem is a Homebrew maintainer. Most of the time, she works on Homebrew in the evenings and weekends when she’s finished with her day job. She enjoys working on it and has been getting more and more confidence to make bigger changes.
Reem noticed that the tool Bob was using was badly outdated and the upstream software providers no longer supported the versions that Homebrew was distributing. She’s concerned that this may expose Homebrew users to a security vulnerability so she posts an announcement in the Homebrew issue tracker, discussion forum, Slack and Twitter to let all Homebrew users of the tool Bob was using know that there are going to be some breaking changes coming up.
Fast-forward a few months and Reem has made these breaking changes and, until today, there had been no issues reported with it. The other Homebrew maintainers said “well done” on her work but no users have commented positively or negatively on the 10 hours of evenings and weekends she’s spent working on this issue.
She sees the new issue opened by Bob. She expresses sympathy with Bob’s position but closes the issue pointing to the deprecation notice and upstream changes and said that Homebrew cannot revert this change now without breaking things for even more users.
Bob is incredibly frustrated with Reem’s response. He is already stressed at work and can’t believe that she has been so dismissive of his needs. He insists the issue be reopened and someone fix these problems.
The issue is not reopened. The behaviour remains unchanged. Bob and others at his workplace make the necessary changes to adapt to the changes in the tool.
The above example has played out hundreds of times in my time working on Homebrew. Hopefully, I’ve made it nuanced enough to make clear that I don’t think either Bob or Reem are bad people or are coming from an unreasonable place.
Unfortunately, Bob’s entitlement is very detrimental to the Homebrew project, Reem and the open source ecosystem as a whole.
Bob and TechCorp are benefitting from the work done in Reem’s free time and from the fact that they’re able to use Homebrew, and the tool installed from it, free of charge in a way that works for them 99% of the time. Additionally, Homebrew’s MIT open source license, like almost all open source licenses, clearly states that Homebrew provides no promises of support, warranties or guarantees that the software will work as expected.
The lifeblood of volunteer-run open source projects, which is most of them, is ultimately the motivation of the maintainers who work on them. Some are doing it in their free time outside of full-time employment or education, some are doing it as part of their full-time employment and some may receive sponsorship for their work. Regardless of their status: they can generally choose what they work on, when they work on it and which issues they decide to fix.
The general state of the open source ecosystem is that most maintainers are building software they want other people to use and find useful. When they break the software in a way that makes it no longer usable, they will generally try to fix this breakage. When users have complaints, they will generally try to alleviate these.
The problem is that these generalities turn into expectations of behaviour, those expectations turn into entitlement and entitlement turns into toxic behaviour that makes maintainers quit open source.
With examples like the above, I have had the (dubious) benefit of being on both Bob’s and Reem’s end of this story multiple times. Many things could have been done differently, so I’ll spell them out separately. Bob in leadership could have:
There are probably more options here that could resolve the issues with these interactions in different ways. The thread that links them all is that the maintainer is offering a free gift of the open source software to Bob and TechCorp but actually keeping that software efficiently running is up to them, not Reem or Homebrew.
The story I provided above is one where Bob’s, and indirectly TechCorp’s, entitlement to Homebrew and the unnamed tool results in demands on the time and emotional energy of Reem and the other Homebrew maintainers. I would be happy if a maintainer I was mentoring responded how Reem did above, that’s one of the best-case scenarios. She was under no obligation to offer a deprecation period, fix issues or express sympathy but did so anyway.
Sadly, maintainers like Reem are often pushed into a situation where they feel obligated to sacrifice their time and mental health to satisfy folks like Bob and TechCorp because they “feel bad”. I want to be very careful to avoid victim-blaming in this situation but there are some resources I’ve personally found helpful dealing with being in a leadership or maintainer position in a widely used open source project for over a decade.
Firstly, you should only be maintaining open source software you use yourself. This is partly because you can’t be a good maintainer unless you can empathise with your users but also because your open source work should be enjoyable and you’re unlikely to enjoy satisfying the demands only of others and not yourself.
Secondly, remember and really deeply internalise the fact that you can stop working on any open source project at any time. Open source maintenance is a job and, like a job, when it stops working for you and you have better options, you can go elsewhere. You should not feel bad about this; instead, you should feel good about the fact that every contribution you continue to make or made in the past were good deeds given freely to others. When your life or the project means it no longer feels good to work on it anymore — stop.
Third, your time as a maintainer is simply more valuable than that of your users. In Homebrew’s case, there’s 30 maintainers and millions of users. It does not scale to prioritise user time over maintainer time. In many, perhaps most open source projects there’s a single maintainer. As a result, this maintainer should only be doing the tasks that users and other contributors cannot do. Alternatively, this maintainer should only be doing the tasks that they want to do.
Finally, maintainers need to learn to say “no” again and again. No to new features. No to breaking changes. No to working on holiday. No to fixing issues or merging pull requests from people who are being unpleasant. No to demands that something has to be fixed right now.
Open source software is a wonderful thing that many individuals and organisations are heavily reliant upon. Much of it is built on the backs of volunteers, though, and all the folks involved with its creation could decide to walk away. As a result, we need to be careful to ensure that maintainers in open source projects maintain boundaries to avoid burnout and users of open source projects avoid entitlement and learned helplessness to solve their own problems.
This article was originally published on WTF is Cloud Native.
]]>We knew everything that needed doing, and we did it. It turns out, whenever you think you know everything: you actually know nothing. If you believe you have a great memory, you probably forget to do things all the time. Any project that assumes a perfect understanding of the existing systems or your superiority over those that built legacy systems will fail.
(G.K.) Chesterton’s Fence provides a great way of thinking about this. If you see no reason why a legacy system works the way it does: you are not allowed to change or remove it. Until you can explain why something was once necessary: you are not in a place to say it is not needed today. If you can’t explain why but need to change or remove it: tread carefully and remember your ignorance.
We did things right the first time without making mistakes like those idiots before. It turns out that when you aren’t willing to make mistakes: they will be more severe and public.
The Cautionary Tales podcast used two great examples to illustrate this. The Billy Joel musical was a disaster in previews, but: they learned from this, and it was a sensation on Broadway. The MacCready Gossamer Condor was the first human-powered aircraft capable of the criteria for the Kremer prize. The Condor did not win from a perfect first attempt. The Condor won because it repeatedly failed, iterating and improving each time.
Building software is the same. Don’t try to avoid failing. Try to fail faster and improve faster. Try to fail earlier when it’s just personal embarrassment in front of your coworkers and not your company’s embarrassment in front of customers and competitors.
Remember that some failures will only happen at production levels of system load. Try to get a high system load without customers relying on it. Even after all this: things will fail. You are incompetent, but know you are and plan accordingly.
We had all the needed resources and more; we had complete buy-in from leadership. It turns out that this made the team unwieldy, and the scope exploded to justify the headcount. Instead of trying to get more and make things better, we should have kept things smaller.
Tiny ships get to production quicker where actual customers can test them. Tiny teams build closer bonds, have more shared context and cannot silo. Tiny scope forces you to think about only what’s essential.
You’re calling that a MVP? It’s not minimal at all. Not being minimal means it might not get released in any form, to anyone, ever. An insignificant project is a project set up for success.
This project was the worst. The timelines were aggressive from the outset. Compromises were necessary everywhere. We had to use that existing system that was unstaffed and in maintenance mode. We didn’t know how the existing systems worked, so we had to rely on trial and error. Failures kept happening, and we didn’t know why. We didn’t have as many engineers, designers or product managers as we needed. We delivered something. Customers loved it.
]]>I’ve been consuming a lot of Brené Brown’s podcasts and read one of her books recently, and they’ve got me thinking about what’s at the root of being able to say “no” regularly and well.
Brown uses the BRAVING acronym to refer to the elements of Trust. Out of these, the ones that jump out to me for saying no are boundaries, reliability and integrity. To meaningfully trust others on these axes, you cannot be someone who says yes to every request. In doing so, you maintain no boundaries, professionally or personally, and will not be a reliable coworker.
To the software engineers reading this, I’m sure we’ve all worked with some APIs that have been great and some that have been terrible. An API with inconsistent and unpredictable behaviour with the same inputs is terrible. A poorly documented API is terrible. An API that attempts to accommodate your request and consistently times out rather than failing immediately is terrible.
Human relationships are like APIs: consistent and predictable responses make them more pleasant. A bad API for your coworkers is when you’re someone who gives radically different estimates for the same work based on whether you want to impress the person or not. A bad API for your coworkers is when you state you’re on holiday or sick but still appear in conversations and meetings anyway. A bad API for your coworkers is when you tell them a 19:00 meeting is fine, but you’re hangry and irritable because you’ve missed dinner and your children’s bedtime.
A previous performance review of mine from a great manager started with the sentence “Mike is very strict with his boundaries”. “Oh no”, I thought to myself, “I’m going to get criticised for always prioritising being there for my children’s dinner and bedtimes four nights of the week”. Instead, he went on to say that this made me easier to work with, set an excellent example for other members of the team to not overwork and set expectations for new parents on the team. Ironically, in this case, being a little more selfish with my time and strict with my boundaries produced an outcome that benefitted the entire team.
Having these boundaries clear and upfront is more complex but crucial for unexpected tasks. If when asked for an estimate, you can tell there’s a bunch of complexity that the requester did not consider and know that your realistic one will disappoint them: you should give them the realistic estimate. If you already feel overwhelmed or have too much to do when asked to do something new: you should say “no” regardless.
I call this “front-loading disappointment”. It’s better to have a conversation about your full workload or longer estimate now when people can prepare for it. Making them happy now and then disappointing them in a few months when you don’t deliver what they planned for you is worse. I’ve also found it less disruptive in my workplaces to ship a project earlier than planned rather than later than planned. If you’re the type of person who hates disappointing others: you’ll need to take my word for it that there’s reduced disappointment overall when it’s pushed earlier. Similarly, you’re not responsible for others disappointment with the facts they didn’t know before you (nicely) revealed them.
Ok, we’ve touched upon why you should say “no”, so let’s discuss when it’s appropriate to do so.
What’s your current workload right now? How are your stress levels outside work? Do you have many commitments you need to meet soon to coworkers, family or friends? What are your goals for the next 3-12 months and what’s the best use of your time to achieve them? What tasks do you have on your plate that are “essential” rather than “nice to have”? Regularly asking yourself these questions can help prime you to remind you of them when receiving a request.
When a new request comes in, and you’ve asked yourself the questions above, you’ll probably have a good sense of whether you can meet the needs and expectations of the requester or not. If you think you cannot right now: say “no”. Every “yes” is implicitly a “no” to something else you might not know yet.
Don’t worry about if you’ve been saying “no” a lot recently; this is not necessarily a problem. Of course, if you’re saying no to every request from coworkers, friends and family, this could be a symptom of a broader problem. At work, though, it’s expected that there will be more demands on your time than your available time. If you’re saying “no” a lot, but you’re still delivering great work, and your coworkers and manager are happy with you: you’re doing enough.
There’s still a little bit of art involved in saying “no”. It may be that your “no” actually means “no because …, “no unless you …” or “no if this …”. These “no”s may be conditional on you as an individual, the timing of the request, the scope of the request or many other reasons. Try to share your reasoning with the person who has requested this of you; it may be that something you understood as critical to the task is actually optional and can be removed given the extra context of a “no because …” or “no unless …” provides.
At least in software engineering, one word to avoid is “impossible”. There are very few things that are genuinely “impossible” (e.g. the halting problem) and avoid it unless it’s one of those. If you’re saying you cannot do a task now, avoiding “impossible” ensures that you don’t try to generalise that to state that no one can do it ever.
If you’re feeling like you’re saying “no” a lot and this is dragging you down, you can ask others to help you. The engineering/project/product manager or a staff-plus engineer on your project may be able to say no on your behalf. It may also be if you repeatedly state “no” to the same person that they are asking the wrong layer of the organisation, i.e. an individual contributor rather than their engineering manager or product manager.
Finally, a “no” is generally a disagreement in either values or information. Figuring out which one applies now may help you figure out how best to communicate or get past the “no”. For example, my current GitHub organisation (Communities) has tight values alignment. When we need to communicate a “no” in Communities, it’s a matter of sharing missing information rather than resolving a values difference. In open source, my “no”s are often due to a misunderstanding about what my obligations are (not) to those people who use my projects.
Thanks to @nerdneha for talking through this post with me and to my long-suffering coworkers and contributors to the open-source projects I maintain for all my “no”s over the years. Thanks to @kenyonj and @katestud for providing excellent feedback on this post.
]]>When estimating a software project you should not:
You cannot increase all your estimates due to all the above. They won’t all be relevant every time. You should ask yourself which of these are most likely to be relevant when you do your estimations. If things change during or after your estimations, e.g. a team change or re-org, you should communicate that the estimates will be affected to whomever you communicated these estimates to.
Of course this article bears a huge debt to the many decades of writing on software estimation such as The Mythical Man Month, which I’ve never actually read, and Peopleware, which I read and enjoyed.
Thanks to @bigkevmcd and @DW-DW-DW for reviewing this article.
]]>You: “I’m looking to hire a senior engineer. Know anyone good?”
Me: “Oh great, I know some good people! How about
$A
?”You: “They won’t do. We need someone with at least
$X
years in$TECHNOLOGY
.Me: “Why?”
You: “They need to be able to get up to speed quickly.”
Me: ☹️
There’s many assumptions being made here that may or may not apply in your organisation:
$TECHNOLOGY
for $X
years means new hires will be able to get up to speed with your codebase using $TECHNOLOGY
because they won’t have to learn $TECHNOLOGY
on the job$TECHNOLOGY
is so perfectly idiomatic that it will be trivial for anyone with $X
years experience in $TECHNOLOGY
to understand the code$TECHNOLOGY
rather than learning your version control system, deployment system, company engineering culture, etc.$X
years in $TECHNOLOGY
that will want to work at your organisation for the compensation you’re willing to pay and loosening these constraints will not get you a better hireYou also may or may not have thought about the following:
$TECHNOLOGY
e.g. databases, on-call, deployment, mentoring, breaking down issues, collaborating with management/product/design/sales etc.$TECHNOLOGY
can often pick up new ones extremely quickly$TECHNOLOGY
and different $TECHNOLOGY
paradigms can often make engineer better at writing all languages$TECHNOLOGY
may not be the best fit for the problem and an engineer with experience in other languages may help you consider future alternatives$TECHNOLOGY
in the medium/long-term$TECHNOLOGY
may not be something that many people have $X
years experience of (but are willing to learn)In my experience, the combination of those assumptions and not thinking about all the above (and more!) mean that I consider it to be an anti-pattern to require experience with a specific technology from engineers. You can factor it into a hiring decision: given two otherwise identical candidates the one with more experience in this language may be more desirable.
If you reword your job advertisements to remove this $TECHNOLOGY
as a “requirement” and replace it with a “desirable” (or just mention you use it) you will find yourself able to get more, better candidates into your hiring pipeline (particularly from underrepresented groups in technology that will often not bother to apply when they don’t meet “requirements”).
This may require changing your interview process. If the majority (or entirety) of your process assumes working familiarity in a given language and you expect the candidate to be able to write that language without any help: the candidate and interviewer are going to have a terrible experience. Ideally, your process allows the candidate to bring a language they are comfortable with and/or is an “open book” pairing process on the technology you use internally. Additionally, the more experienced the candidate or more senior the position: the more of the interview process should be dedicated to communication, engineering best practises, architecture, etc. rather than just programming.
This can be a hard adjustment to make but it’s one the best engineering interviewers made years ago (and you can too).
By “a senior-plus experience” I mean e.g. senior, staff, principal engineers etc. Never heard of staff or principal engineers? Check out my “What is a Staff (or Staff-Plus or Principal) Engineer?” article .
]]>When most people hear the term “economics” they tend to think about how 💵💶💷💴 flows around an economy and, particularly in a capitalist economy, how the allocation of capital affects the throughput of businesses operating in a free market.
As a result, when people hear the term “open source economics” they tend to jump to the same conclusions: it’s about how 💵💶💷💴 flows between and is invested in open source projects. This is not a bad conclusion, it results in the creation of tools such as GitHub Sponsors that benefit many open source maintainers but it’s not the whole picture.
When you look just at the financial side of open source projects you tend to assume any problems they suffer are due to a lack of investment and can be solved by adding more 💵💶💷💴. For example, if a project struggles to make releases, review and merge pull requests, close issues or answer discussions: if they had more money they could pay someone to do that work and more of it will get done.
Unfortunately, money alone will not always fix any of these problems. Money can help but it needs correct investment and understanding of what the real problem is.
An “economic problem” is one which requires the allocation of limited resources in order to solve a problem. In modern, capitalist economies: the limited resources are usually 💵💶💷💴. Where there is a labour constraint, this is solved by paying enough money to compensate or attract the right candidates to solve the problem.
Open source software is a little different because of small pool of “labour” (maintainers) with knowledge of a project (perhaps single person) most of whom do their work voluntarily for no direct financial compensation. Many widely used open source projects are built by a single maintainer and, if so, this maintainer is the only person who can make all releases, review and merge all pull requests. Even with multiple maintainers, a widely used project will have a much smaller number of people doing this work than consuming the results. This means that domain knowledge and access control is limited to this one person or limited group of people.
The Open Source Contributor Funnel is pretty stark. In Homebrew’s case: there’s millions of users, thousands of contributors and tens of maintainers.
Relatedly, the time that these maintainers spend on the project may not be optimally allocated for the continued progress of the project. It’s easy for their time to become monopolised by niche problems from a few users that require a lot of individual attention at the detriment of writing code to improve the project for the majority of users. This makes sense from the incentives of each user to just get their problem solved but not for the project as a whole (and may not even be the best for the user if it slows or halts feature development).
Relatedly, this is why you should stop mentoring first-time contributors.
A tragedy of the commons occurs not from consumers over-appropriating the content itself, but from consumers over-appropriating a creator’s attention.
Nadia Eghbal, Working in Public
As a result, the “open source economic problem” is solving how to have sufficient labour that is allocated efficiently. Money can be part of this by e.g. increasing the ability or motivation of a maintainer to spend more of their time on the project but unless this is allocated effectively the project may not be any better off.
There’s also some thresholds effects for a project’s income. For a project run by a single maintainer with a day job, if they are paid enough to quit and work full-time on open source that may hugely increase the throughput of the project. However, even at 5% less than the amount they require to quit, they may not be able to dedicate any more time to the project. In some situations they may be able to pay others to take work off their plate but not always.
Viewed through a labour-centric rather than money-centric lens, the “more money invested in a open source project translates directly to a more effective project” obviously falls short. However, there is a place for maintainers and sponsors of open source projects to solve these problems.
There are some aspects of running an open source project that can only be done by maintainers (or their automation). These include merging pull requests, making releases and closing issues. Without new commits, merged pull requests or releases: a project is effectively dead.
As a result, maintainers should focus their time on what only they can do and delegate as much as possible to automation and community members.
For some (GitHub-centric, it’s where I’ve done all of my open source work for years before I worked there) examples:
Finally, and most importantly, maintainers should primarily focus on what they most enjoy doing. In all open source projects (and particularly those that are mostly volunteer run): the motivation of the maintainers is what keeps the project running. Remember: open source maintainers owe you nothing! If the maintainers spend most of their time doing things they don’t enjoy, this motivation will deplete and eventually expire and the maintainer will abandon the project, perhaps with no-one to replace them.
As mentioned above, when a sufficient threshold is reached, a maintainer may be able to go full-time to work on their open source project(s). If the maintainer has stated this as a goal publicly, helping them to reach it by spreading the word may be tremendously valuable to the project.
This may not be their goal, though, many maintainers are happy with their day jobs and would rather top-up their existing income or spend the money elsewhere. Sponsorship of open source projects is particularly useful when it helps to save time for the maintainers e.g. software tools, infrastructure, etc.
Although it requires a non-trivial investment of time from the maintainer, spending money by bringing more maintainers into the project through programs such as Google Summer of Code, Outreachy or Major League Hacking Fellowship can help add more contributors (and hopefully maintainers) to the project and reduce the burden on the current maintainers.
Money is an important ingredient into improving the open source ecosystem but it must come along with prioritising the motivation of maintainers, how their time is spent and realising that it does not solve all problems.
Thanks to Ron McQuaid, Denise Yu, Neha Batra and Alan Donovan for reviewing this post and providing helpful feedback.
]]>Before GitHub, the engineering career track I have been most familiar with looked something like this:
At GitHub in 2021, the 1st to 3rd is the same, with different terminology I’ll gloss over, but there are three possible choices:
The EM and IC tracks run in parallel, so you have something like:
Generally, the reporting structures are across a level so that you can expect a Senior Engineer to report to an Engineering Manager, Staff Engineer to a Director of Engineering, etc.
The compensation scales and expected areas of influence for each level are the same. For example:
Similarly, Principal Engineers’ output is expected to affect the entire company and Distinguished Engineers’ output across more than just the company (i.e. the industry).
The difference in responsibilities between an Engineering Manager and Senior Engineer is generally well understood. An EM should have regular 1:1s with their direct reports, will provide their performance reviews and, depending on the organisation, may provide some technical leadership, product management, project management, or even coding beyond their primary people management responsibilities.
A Staff/Principal/Distinguished Engineer should have no people management responsibilities whereas their EM/Director/VP counterpart will find that taking up a significant part of their role. The Staff/Principal/Distinguished Engineer will take responsibility for technical leadership, mentorship and is likely to still code and, in some cases, spend the majority of their time still doing so.
My staff engineer role changed from primarily feature coding work to engineering-wide work and feature non-coding work (e.g. scope adjustment, review, architecture). Increasing non-feature, time-critical commitments (e.g. mentoring, code review outside of the feature’s area, on-call, support existing staff projects, performing low-context fixes for my manager) caused me to be slow to complete feature coding work. After discussions with my manager, we agreed it was better for me to no longer be on the critical path for features.
As I embraced my staff engineer role, I found myself fitting into the “Solver” (and somewhat “Right Hand”) archetype. I am more useful as a value multiplier for other engineers rather than coding in areas where I have less context than other engineers.
I now optimise for unblocking others, being a “💩 umbrella” for other ICs, taking on high-urgency but low-context work and focusing on development experience for engineers at GitHub.
As I’m not part of a specific feature team, I fill out a weekly Geekbot report in Slack and keep track of my work in my GitHub project board. This increases the visibility of my work to others, stops things falling between the cracks and makes it easier to write my self-review, part of our performance review process.
When I became a staff engineer, I expected that I’d be doing less feature work and less coding. I probably code a little less than I expected to but I am happier with this than I expected to be.
I’ve been consistently and passionately disinterested in going into engineering management because I still like doing non-trivial amounts of coding and being evaluated based on that sort of work. Being a staff engineer has been a significant improvement for me for where I’m at in my career, family life and geographical location/timezone.
Despite it being a relatively well-trodden path in the technology industry at this point, there are limited resources to learn more about being a Staff+ engineer. Those I’d recommend are:
#staff-principal-engineering
channelThanks to Graeme Arthur, Sarah Vessels, Keith Duncan and Luke Hefson for reviewing this post and providing helpful feedback.
]]>On the mentee side, I did most of my mentorship initially outside GitHub through mentoring new Homebrew maintainers informally and then more formally through Google Summer of Code. I’ve been lucky enough to work with folks such as @itsbagpack and @gallexi through their promotions to staff and senior engineer respectively; helping them figure out their role, strengths and weaknesses, review work and documents for them and generally have fun hanging out with inspiring engineers like them.
My situation shouldn’t be special, though!
For almost everyone in our industry: there is someone around who has more experience than you. That person will have some things you can learn from them.
Additionally, there’s lots of people in our industry who you may have more overall experience than but they have more experience in a particular subject or skill that you wish to learn more about.
For almost everyone in our industry: there is someone around who has less experience than you. If you’re new to the industry, you could mentor someone who is e.g. in a bootcamp, studying at college or high school.
The mentor and mentee relationship is important but to remain grounded and better understand yourself and what skills your team most needs right now you also need to remain connected to your peers. Doing so may already happen organically through pre-existing relationships, video calls, group chat channels etc. If it isn’t, though, or if you wish to supercharge this process: having regular 1:1 video conversations with your peers (structured or unstructured depending on your preferences) can help you improve this.
The above approach to mentors, peers and mentees can be summarised by this “mentorship diamond”:
Let’s look at a few job titles as examples:
If you’ve already got someone who wants to be mentored by you: great! Discuss with them what cadence would work best for you both (weekly, fortnightly, monthly), whether you’d like it to be a structured or unstructured conversation, whether you’d like to pair together, etc.
If you haven’t already got someone who wants to be mentored: ask around. Ask your manager and other managers around the company if they have someone that can benefit from your skills. Consider looking outside the company for less experienced people you may have relationships with that you can offer mentoring too. If this is too intimidating, consider a structured program like e.g. Google Summer of Code.
Mentoring relationships also aren’t always “official”. They may be informal conversations with someone you’re regularly chatting and sharing knowledge with.
If you don’t have the inclination or bandwidth to take on a(nother) mentee right now: that’s ok! You can politely decline and, if you’re feeling extra nice, help them figure out someone more suitable.
This is a bit easier: figure out who you’d like to work with as a mentor and ask them if they can spare the time to be your mentor. Not everyone I’ve asked has said yes but I’m always glad I asked.
If you’re not sure yet who would be the best person to work with or you’ve asked people and they’ve turned you down: ask your manager for suggestions.
You can also start with having just informal one-off conversations with someone you would like to be a mentor and, if it goes well, arrange them to occur regularly.
This article written by @nerdneha is a great resource when searching for a mentor.
Thanks to @nerdneha for suggesting, reviewing and generally inspiring much of this post. Thanks to @seejohnrun and @eileencodes for being great mentors and @itsbagpack, @gallexi and many others in and outside GitHub over the years for being great mentees and reviewing this post.
Firstly, when someone asks me to do something I immediately write it down in my Apple’s Notes app. This doesn’t have any mechanism for storing dates or times but just lets me enter text. This is good because it’s available and synced everywhere. If I have time (or when I next do) I then sort into lists based on topic matter. Some of my long-running lists are: GitHub (my current employer), Open Source (encompassing Homebrew and open-source projects I work on) and Random (a short dump for other tasks). Every time I do work for GitHub, open-source or myself I look on the relevant location to see what I should do.
I try my long-running notes structured into sections headed:
When there’s a task with a time or date I put it into either Apple’s Reminders or Calendar app. Reminders gets anything that needs done on a particular date or repeatedly but doesn’t require me to go anywhere. Calendar is generally when I need to go somewhere or if it’s e.g. an online meeting that has a specific start and end time. These apps will pop up alerts on my Apple devices with a noise or vibration so I actually remember them.
The above was sufficient in 2015 when I first wrote this post but since I got promoted to Staff Engineer at GitHub I’ve had to get a lot better at keeping track of what I plan to work on in future, am working on right now and have completed. I’ve found a personal GitHub Projects board to be a good fit for this. I have columns headed:
If you have any familiarity with Getting Things Done then some of the above may seem familiar. I tried to follow it strictly in the past but found the structure overkill for me personally. I tend to keep my various TODO lists short enough that I can quickly scan and mentally prioritise them daily.
So far it’s relatively straightforward but I’ve omitted the most common productivity nightmare: email. I’ve heard the main problem with email being that it’s basically a TODO list that anyone can add to at any time. This is a problem.
First thing to do with your email is trying to keep Inbox Zero. In short: the default state of your inbox should be to have no emails in it and when you check your email you should end up with an empty inbox afterwards. This sounds like a bit of a pipe dream so let me tell you how I try to do this.
You probably get a bunch of email that you expect. Someone commented on your GitHub issue. Your company’s weekly newsletter got sent round. That person in your office who sends you only junk. These all patterns and patterns are good because a computer can handle them trivially for us. What I do is set up filters (in Gmail, iCloud, Outlook, etc.) which sorts each type of email into a folder (“label and archive” in Gmail). My long-running personal email folders are Bills, GitHub, News, Social (all social media notifications) and Software. This immediately let’s me prioritise these groups. Anything in these folders is never urgent during work time. If I’m emailed by someone I didn’t expect (e.g. my dog) it may be urgent so it ends up in the inbox instead of a folder.
I also keep my personal, Homebrew and GitHub emails in separate email accounts with different filtering rules. This makes it easier for me to ignore Homebrew and GitHub emails when I’m not working or on my phone. If you’re interested in my GitHub filtering rules, I wrote a post on the GitHub blog about managing large numbers of notifications.
It’s important to ask yourself when you get a new email: was this expected, somewhat urgent/important or even desirable? If not, filter it into a folder, unsubscribe from the mailing list, tweak your social media notifications or setup a filter to just automatically mark it as read and archive/delete it. Stop wasting your time repeatedly ignoring the same emails.
The tricky thing with the above system: what do you do if you don’t have the time to reply immediately to an email or it’s generally something you want to deal with later? I used to enjoy the Mailbox (an email client from DropBox) “Later” feature which could remind me of an email at a later point. Mailbox is no longer around and I’ve been burned too many times by my email clients going away that I feel I can no longer rely on this functionality (until it’s in Apple Mail or Gmail).
I now have returned to will leaving email in my inbox if it needs actioned imminently (i.e. this month) or flag (“star” in Gmail) if I need it at some point in future. Both Apple’s Mail app and particularly Gmail provide good enough search that I can always find what I need (as I always archive and never delete emails).
Slack has become a bigger part of my life for GitHub and Homebrew since I originally wrote this post in 2015. I tend to treat it pretty similarly to email: it can be a bit of a TODO list but I don’t want others to control it. Thankfully, it does have a pretty decent reminders feature that gets me back what I lost from Mailbox.
I hope you found this post to be useful. To summarise: immediately write down tasks you need to do on your phone/computer, triage them regularly, assign times/dates when possible and automate your email for easier prioritisation.
Finally, a pet hate: if you commit to doing something for someone please immediately make a note of this and actually do it in a timely fashion or let the person know if you can’t. This alone will make you seem like A Productive Person. Don’t try to just remember (because you won’t) or write it on a scrap of paper because you’ll lose it.
Good luck!
]]>I’ve written before about how Homebrew’s CI system has evolved over time but not what I’ve had to learn to make it work as well as it does.
In the earlier days of Homebrew all review and testing was manual. This involved a maintainer checking out a GitHub pull request onto their local Homebrew installation and verifying it worked as expected. I’m obsessed in my personal, professional and open source lives in reducing the friction to perform common tasks so my first commit to the Homebrew package manager (i.e. not a package) was to add a brew pull
command to simplify this process.
When maintainers found problems in a pull request they reported back to contributors, contributors fixed them and the pull request got merged. Contributors did not expect a merge of their contribution unless it worked, after all. Where it got tricky was when a change “worked” but violated a technical standard. Often when giving feedback in these cases there would be accusations of pedantry or personal preference.
The first attempt at addressing this was the addition of a brew audit
command by @adamv (the first Homebrew maintainer that wasn’t @mxcl, the creator of Homebrew). This command, simple at first, was turning lessons learned from code review into a command run to verify correctness. Maintainers and contributors ran this command on their local machines and contributors accepted that a warning flagged by this command was not personal preference but a codified standard.
This process still required a bunch of manual intervention from contributors and maintainers; what we needed was to automate this process. To meet this goal I started and ran a successful Kickstarter project to fund CI machines for Homebrew. After I’d set up these machines, the automatic brew audit
of GitHub pull requests provided much quicker feedback to contributors.
I started to notice that when we turned “pedantic” comments by maintainers on pull requests into brew audit
checks run locally and by CI, contributors ceased accusations of pedantry and usually complied without argument to the requests of the tool, often without any human intervention being necessary.
Over the years we’ve tried to turn all repeatable review comments into code checks on pull requests using brew audit
and tools like RuboCop to enforce consistent Ruby code style. This has now even enabled us to run brew audit
checks directly in your text editor. This avoids different standards between maintainers and the previous arguments about pedantry: developers accept that robots are by definition pedantic and consistent so don’t try to convince the robot to make an exception.
It’s hard to state how much time it saves to not have arguments between maintainers and contributors on pull requests on style preferences or technical minutiae. Instead these discussions are on the original implementation of the rules instead of their application.
I’ve been glad to see this pattern expand to many other GitHub projects. GitHub Actions has become an easy way to enforce checks onto your project. A particular favourite of mine is the actions/stale which closes issues and pull requests with inactivity.
What robots can’t do is empathy and building human communities. On a project like Homebrew that receives large numbers of contributions from large numbers of people, it’s easy to overlook the work of the individuals that goes into it. A longer, heartfelt message on a less busy project becomes a “Thanks” on a busier one.
I wanted to address this but at the same time recognised that this isn’t something to automate; a heartfelt thanks or welcome from a robot doesn’t have the same degree of positive emotional impact as coming from a human. I wanted to make it as frictionless as possible to express my gratitude.
My first implementation of this was using TextExpander to allow me to e.g. type ;thx
and have it expand to Thanks so much for your contribution! Without people like you submitting PRs we couldn't run this project. You rock!
. This can then be further tweaked to add their name, a note about their specific contribution or if it’s their first to further add a human touch. People have told me both online (and in person when I’ve met them at conferences) that this made them feel great about their contribution and made them want to contribute again. They still felt this way even when I told them how I did so because they understood that my underlying gratitude is not proportional to the time I spent typing a message. A robot messaging them would not have had the same positive effect.
As I work at GitHub I’ve been able to further improve my process to add keyboard shortcuts to GitHub’s saved replies which means that I don’t need other software and can do this all in GitHub.
My main takeaway from this has been that you should seek on your project to automate as much as you can (particularly turning documentation into code), while remembering that the human touch is still necessary to praise and be kind to your contributors.
Let robots 🤖 handle your project’s pedantry and humans 🥰 handle your project’s empathy.
]]>Here’s the categories as I currently see them:
I’ve only ever worked for 4. (KDAB, AllTrails, GitHub). Very early days at GitHub felt like some individuals were in 5. (but it was rare).
]]>One I picked up from Max Howell, the creator of Homebrew.
When you’ve got a non-empty issue backlog on your open source or commercial project: don’t waste your time “fixing” things that aren’t problems affecting users. Yes, refactoring that method so it returns an empty string instead of nil
feels useful but if there’s no way that bubbles up to a user issue: you have better things you could be doing with your time.
A solution that’s “good enough” (or even “better than the status quo”) and can be shipped this week is always superior to one that can ship next week. Iterative improvement allows you to learn more every time you ship and encourages you to keep shipping.
Additionally, in both commercial and open source environments I’ve seen motivation to fix a problem today destroyed by promises of some big solution that will fix this problem (and many more!) in six months time. Relatedly: these big “fix everything” projects almost never end up shipping and then the “quick win” fix has been long abandoned.
One from my Dad: it costs nothing to say sorry and it’s worth saying even when you aren’t convinced you’re in the wrong. It’s an easy way to demonstrate humility, make other people feel better and have someone take accountability for mistakes rather than no-one.
Chesterton’s fence, basically. Try to make your changes to methods/files/systems as small as possible in order to accomplish what you want to.
Refactoring for refactoring’s sake (“I’m refactoring because the code is hard to read” often is “I’m refactoring because I didn’t write this code”) is an example of this in action; it often introduces new bugs and provides no direct benefit to the end-user.
Of course, if the code is unspeakably messy/buggy so that it takes forever to do any new development: refactoring may be in order. Just don’t do it because “this looks messy”.
DRY is a pretty obvious and basic engineering principle at this point but I like to take it to new extremes:
I’ve written previously about how open source maintainers owe you nothing and this is essentially the same thing.
Motivation isn’t the same for everyone. For some, it’s financial (hence my work on GitHub Sponsors). For others, it’s about having fun (so blocking unfun, rude people makes sense).
When maintaining or developing open source: don’t be drawn into doing things you don’t want to do.
When using/contributing to/submitting issues for open source: don’t try to make others do things that they (or, as a shortcut, you) don’t want to do.
I’ve written previously about how I get things done but this is the cornerstone.
Anyone I know who does this is generally fairly organised. Everyone I know who doesn’t is not. The worst are those who have “a system” which doesn’t involve writing anything down and they feel like they are organised and aren’t. Don’t be that person.
Related to multiple of the above. Doing the most important/useful thing right now means fighting off the other 100 things that 10 people want you to do instead that are less impactful.
This is even more applicable when designing product and soliciting feedback from users/customers. Some of them will have universal ideas and some will have ideas that just a hack around their particular problem today.
If you say no to everything the first time you hear it then you’ll be able to more easily prioritise the things that keep coming up again and again.
People respond way better to pedantry from robots and empathy from humans. As a result, prioritise your time as an engineer on turning your desire for cleanliness and pedantry into automated checks.
Don’t check that list once a month, get a bot to do it for you. Don’t remember to follow a deployment process, turn it into a script/application.
Another one from Max Howell.
If you’re building software that is a tool for people (particularly a developer tool) and asking yourself how some feature should work: ask yourself how it can be most useful. This is often in tension with “be consistent” but often a better experience arises from being less consistent on the occasions where it makes a tool more useful (particularly if inconsistent with rules you’ve created).
This has been a real big change for me in the last 5-10 years and one I’m not consistently good at (and props to Nadia Eghbal for nudging me initially).
You can get a lot of drive-by negativity through open-source (and sometimes even just working in a big company). If you’re someone (like me) who prioritises being responsive then it can feel like you should respond to this negativity with disagreement or criticism. Usually (particularly if you feel angry/sad) the best thing to do is just to ignore it and move on.
]]>Many software projects have a “beta channel” that relies on users to know about it and (hopefully sensibly) self-select into it. This can work well if you have built-in exception reporting in your application, but for Homebrew we wanted a way to segment our users without their explicit intention. Additionally we needed to ensure that the users who received the beta channel were those who were willing and able to interact with the project through good bug reports and pull requests.
Homebrew has the concept of user commands and developer commands for interacting with the application. A user of Homebrew who does not wish to contribute to the project or run their own third-party repository (“tap”) will never need to run a developer command. As a result, the action of running developer command signals that the user is more likely to interact with the Homebrew project and ecosystem and have a better understanding of Homebrew internals or concepts. When a user runs a developer command we set a flag (which they can opt in or out of) which means their future updates will be on the beta channel. Technically this means that commits to ‘master’ are immediately deployed to these users while other users get these changes only when they arrive in a stable tag.
This segmentation results in 0.1% of our users being in the beta channel and 99.9% in the stable channel. We try to ensure that major changes sit in the beta channel for at least a workday before they make it into the stable branch. This has resulted in a more stable experience for most users and excellent bug reports and pull requests from the beta users. A small number of beta users complain when they realise how our segmentation works but are placated by the opt-out.
Sometimes new features are so unstable or unpredictable that we don’t want them to even go to our beta users yet. In this case we use feature flags through environment variables that enable the (new) functionality. These can be enabled manually by particularly brave guinea pigs (usually me), followed by other maintainers and/or our CI system and then go to the beta channel and stable users. This ability to gradually migrate drastic changes through increasingly large numbers of users through trivial runtime changes enables easy experimentation without damaging the stable experience we aspire to provide to most of Homebrew’s users.
Thanks to Issy Long for reviewing this post.
]]>In engineering the one consistency is that you are going to have to work on or with things that are new to you. What differentiates good from the rest is how they handle this unfamiliarity. The good engineers I’ve worked with are cautious with unfamiliarity; they write more tests, read more documentation, code more defensively and ask for help.
Asking for help when you’ve been struggling with something for too long (and read the documentation) shows humility and allows teams to better pool knowledge. Even more importantly than asking on unfamiliarity is asking on mistakes. It’s never nice to admit a failing but the good engineers I’ve worked with realise that their personal reputation is less important than resolving an issue they may have created. They rapidly disclose their errs so they may be assisted by others. This applies more when you are less experienced and work with those with more experience: quickly admitting a mistake may allow them to intervene and prevent an issue from affecting your users.
In software development we rarely have the luxury of a todo list with a single item on it. Despite this, we are not able to work on two things simultaneously. Every time you work on one thing and change to another you pay the cost of mentally (and sometimes in development environment) context switching. As a result the most efficient way to work is generally to take one task to completion before starting another task. Obviously there are times when you may be blocked for long enough that you can finish another task before you’re unblocked but prioritise finishing (or eliminating) a task before moving on.
A good tool I find in doing this is your GitHub “created pull requests” page. This lists the pull requests that you created that are open (i.e. not closed or merged). Every day I like to ask myself: how can I finish (or otherwise close) one of these today? Prioritise doing that over starting yet another one.
If the cause of having to work on multiple things at once is being blocked on input from a coworker, do what you can to avoid this becoming a problem. Firstly, be the change you want to see in the world and make it a high priority to ensure you’re not blocking your coworkers. This means being responsive to your direct mentions or messages on Slack and GitHub and review requests on GitHub (particularly if you have a workflow that requires review before deployment or merge). You may find my “Managing large numbers of GitHub notifications” post on the GitHub blog helpful in this endeavour. It also means ensuring that questions you’re asked repeatedly have their answers documented and tasks you perform that can be automated have automation created.
For your own work, try to ensure that you gather the requirements for a task before you start it. This may involve asking the right questions from the right people before you start coding. Similarly, when you’ve started a task, if you are blocked by another, ask yourself how can use your best judgment to cut scope or build based on your current assumptions and disclose these when you’re finished.
Thanks to Graeme Arthur for reviewing this post and pointing out that avoiding multitasking and proriotising unblocking are tenets of Kanban.
]]>In 2013 Homebrew’s method of building binary packages (bottles) involved me manually doing so in a VM on my Mac. Similarly, user contributions were tested by being run on the machines of the reviewing maintainer. This wasn’t going to scale. I decided the solution was for us to have our own dedicated Macs to run CI on pull requests and build our binary packages.
Homebrew launched a Kickstarter to pay for these Macs and it was completely funded in less than 24h and we made nearly 10x our original goal. The lovely Positive Internet Company also offered to host the new Mac Minis for free so we had a high-speed, reliable connection and host.
This predated the existence of free, decent Mac CI or cloud hosting services (and my becoming a father) so a lot of time had to be invested for initial setup and configuration and ongoing maintenance and upgrades. Most of this fell on my shoulders as I was the main person willing and encouraging the work.
As time went on our financial, legislative and technical CI systems became limited again. A one-off payment to a UK bank account, no legal/non-profit presence and maintenance falling on my shoulders meant we had to look for better solutions.
The Software Freedom Conservancy accepted Homebrew as a member project and we transferred our existing funds to them and, through them, now had a 501c3 to receive charitable donations and a legal entity to control our finances, expenses and nudge us into having some structure (a project leadership committee) to manage these.
With a goal to move more of our self-hosted to cloud hosting (free or paid if needed) and pay for a conference for maintainers to meet one another we wanted to receive regular, predictable donations to the project and notify our users that we needed their donations.
For predictable donations we set up the standard (at the time at least): a Patreon account. We offered nothing in exchange for donations but to told people we were an entirely volunteer-run project.
We’d had a section of our README soliciting donations but this never got too much uptake. We’d previously wanted to make a similar global notification to users about our opt-out analytics. This approach (a one-time message on interactive sessions) felt appropriate for donations too. We show users a one-time message on first install or on a Homebrew update to tell them we needed donations and where and how to do so.
As soon as this message rolled out we saw a huge jump on donations eventually settling between $2500-$3000 a month on Patreon along with some (large) one-off donations through the Software Freedom Conservancy. This provides us with the ability to pay for services and software we need and for maintainers to meet each other for high-bandwidth communication in person. This was achieved without selling our users attention or data (advertising), nagging them regularly (nagware) or making them pay for Homebrew (proprietary software licenses) so resulted in no backlash and a better funded project. Win/win!
Thanks to Nadia Eghbal for reviewing this post.
]]>Where previously I had a fairly relaxed morning getting started at work I now take my eldest to nursery most work days. Similarly, I have dinner with my wife and kids every weekday (bar one where it clashes with a meeting with west coast USA folks) and then spend time with my eldest until they go to bed. I also go to the gym most work days during the day (a bonus of getting up early). What was a general principle about not working in the evenings and weekends has turned into a hard rule for my family and to recharge.
This does not give me much time in which to procrastinate, mess around or work on things (professionally or open source) that are not important. What’s important is now defined by things at work that meet OKRs, help lots of other engineers or make my manager look good and things in Homebrew and other open source that can (or will) only be done by me.
I previously took work email and Slack off my phone but I’ve now done the same for my Homebrew email and Slack too. It was good to be able respond to an urgent Homebrew issue (when awake) within minutes but now that I can’t do anything about it for hours the notifications cause unnecessary stress. Sometimes this means I’ll go all weekend without checking email or Slack which means other Homebrew maintainers need to step up (and have done well, nicely increasing the bus factor). Several other Homebrew maintainers are parents but all of them have been very understanding about my delayed responses to urgent issues.
I tend to now do more of my open source work during working hours when I have downtime or am blocked on other tasks. This might have felt in the past like taking time from my employer but now that I have less time in general (see above) it feels like I’ve got more time “for free” by focusing more and spending less work time on things that don’t matter.
I’ve lost my British Airways silver status due to less long distance business travel and almost no holiday travel. This pained me but was a nice sign that I’m spending more time with my family (which I enjoy). The travel I do do is much harder because of how much I miss my family and knowing it’s harder for my wife while away (even with the help she gets from my lovely in-laws).
I’ve moved my home office from a room in my house into a (separate building) garage conversion and now have the benefits of a separate office while being less than a minute’s walk from any part of my house. This setup plus the flexibility of being able to start work after dropping my eldest at nursery, go to the gym during the day, be home for dinner almost every night means I’m even more attached to remote working than I ever have been (in the 10 years I’ve been doing it). I doubt I will ever go back to another office job (even in Edinburgh, where I live) unless I’m somehow able to maintain the (extreme) levels of flexibility I have now. Relatedly, any new team or company I join I need to be confident that when I take time off to look after an ill child sent home from nursery (as my wife works we share this responsibility) that this will not be received negatively.
I’ve written previously about how open source maintainers owe you nothing but this has become far more engrained for me. I’m far quicker to block people being unpleasant and resolve interpersonal issues amongst Homebrew maintainers because my open source time is more precious and focused. Additionally, previous upsets with people in open source that I’ve just tolerated have made me irritable and detached from my family when we’re spending time together. I’m not willing to put up with that any more.
Many people may read the above and think “wow, that sounds like big transition, I hope it was worth it”. In short: it is. Moments spending hugging a child before they go to sleep are so lovely I literally lie awake at night thinking about them. I’m incredibly lucky and privileged to have the opportunity to have a setup where I’m able to do that when friends and peers are still stuck at (or commuting to/from) work. I regret nothing and will treasure these moments forever.
In no particular order:
What are the open source projects that you regularly contribute to?
The main one would be a project called Homebrew: it’s a Mac package manager. And then, there’s some projects of my own, such as one called Strap, that’s probably the most like widely used one created by me. And other kind of random bits and pieces periodically.
How often do you work on those?
Homebrew - I work on basically every day. And then, the Strap probably once a week, or a couple times a month. And then the rest… again, when it comes down to it, it just varies a lot.
Every day?
Yeah, for Homebrew, every day, pretty much.
How long would you say per day that you work on Homebrew?
Probably zero to two hours.
Wow. Have you ever been paid to contribute?
Not to the Homebrew project. I have been paid for open source work in the past. And not specifically, it wasn’t framed like that, where someone was like, “Hey would you like to contribute this thing for money?” But it’s been, during the course of my employment, I needed to contribute.
Indirectly, I guess I have contributed to Homebrew in that way, and so yeah, indirectly, through my employer, I’ve needed to make changes in Homebrew while being paid.
Right. If you had to say what tasks you were paid to do for Homebrew, or for an open source project?
I guess feature development and bug fixing.
Moving on from that. In your best estimate, how many people would you say actively contribute to Homebrew? So yeah, how many people would you say actively contribute to Homebrew in some way?
We’ve had over seven thousand people, like at all, ever. On a given month, you’re probably talking hundreds of people.
How many people contribute at the same level that you contribute?
That’s a tricky one. So I’m the project lead now, and in that respect, probably no one. In terms of the amount of time they spend on it, and I spend on it, over a longer a period, on a day-to-day basis it’s kind of hard to know in terms of time people spend because you just see the artefacts.
Again, there’s probably tens of people who are spending more than an hour a week, I would say. Less than a hundred, more than 10.
What kind of work do you generally do?
I do a bit of everything, from feature development, bug fixing, triaging, user issues, helping, helping users figure out solutions to their problems, as well as kind of system administration tasks, and then like some documentation. Also what’s almost like people management - directing what people could or should be working on. A certain amount of mentorship probably, as well.
Is there work that you do that you think you’re one of the only ones that does that particular task?
Yes, yes. There’s quite a few, unfortunately, where I’m the only person.
What do you mean by that?
For example, today, to be very specific, we have an automated testing system that we run ourselves. Because none of the stuff available for free or at the market works for our needs, yet. I’m mostly the main person who’s kind of set that up, and knows how it works. So when it breaks, it’s generally me that is involved with fixing that. And so, if I was hit by a bus, people would maybe eventually learn how to fix that, but it would probably be a little bit difficult for people to get there.
But yeah, on a day-to-day basis, when stuff like that breaks, it’s generally me who fixes it.
Okay. I’m curious - what do you think that means - to be able to do a task that no one else in the project is working on?
Sure. I guess it simultaneously feels good, and feels bad in that it’s nice to feel valued, and important, and whatever. But then, at the same time, it doesn’t particularly bode well for the future, which is why, with tasks like that, we’re trying to sort of either eliminate the task, or figure out how to get more people involved in stuff like that.
Because, obviously, it doesn’t work very well if something happens to me. I also have one, and soon to be two small children. So they occupy significantly more of my time than I’m able to spend doing stuff like this.
Historically, I have the Homebrew chat application on my phone, and if someone sent me a message saying something was broken, I’d pretty much drop what I was doing, and go, and fix it. And I don’t do that anymore, because it interferes with family life. So yeah, it’s tricky.
Alright. It sounds like you’re doing a lot of really good work for this project.
Thank you.
That is very heavily relied on by many people.
Yes. Again, which is simultaneously a nice, and not-nice thing.
One of the ways some are starting to frame open source projects is as, infrastructure. How would you respond to that term?
Yeah. If you’ve been at all inspired by Nadia Eghbal, her and I used to work in the same team together. I would very much agree with that description of open source infrastructure. Particularly certain things feel more “infrastructurey” than others. And some open source stuff is very like a product, and it just so happens to be open source. And then some stuff - Homebrew would be an example - where it’s being used primarily, if not exclusively, by developers in order to build other software. So that description feels apt, because it’s not being used for it’s own sake, but in order to help for something else.
Like a stepping stone?
Yeah.
I wanted to ask a question that I should have asked earlier, which was if you have a formalised role in the project?
Yep. So I’m the “project leader”; that’s my official project title. And on that we’ve added a bit more governance and structure. So as of this year that was an elected position, which I was elected to. And I’m also on the project leadership committee and the technical steering committee. The reason why is I guess having formal roles is part of being a maintainer for the project.
How did you get those roles?
All of those roles are now, as of February, elected positions. So I was elected to each of those roles.
How do they have elections?
So we’ve only done one so far. So basically we had and are going to have in the future a annual general meeting. I can send you, if you’re interested in some of it, we have all written up basically how the governance process all works.
That’d be great, yeah.
See: Homebrew Governance.
Okay. So I wanted to talk more specifically about some of the work that you do. First, how do you decide what you spend your time contributing to?
It’s a combination of what I find interesting or fun and what I find important and things that I think only I can do. So things may lean more in one direction or another. Ideally it is something which is in all three of those categories, but then if it’s something where I know a bunch of other people can do it, but I quite want to and I’ll find it entertaining to do that, then I may well do it. And in a similar way, there’s a bunch of stuff that, I mentioned earlier, which only I can really do at the moment and I would rather I didn’t have to do that stuff. My day job is pretty far from that and no one else can do it right now, so I kinda have to.
How would you say the immediacy of needing to fix things plays into that?
Yeah, good question. I think that certainly defines what I choose to work on. This particular issue we had today, for example, I have kind of a longer-term project to try and replace this stuff. But then a bunch of higher priority things keep getting inserted above there in my to-do list. So it becomes harder to get around to that. So I’m probably going to, in the next month or so, take like a week’s vacation and just try and focus just on doing this and nothing else, so I can get it off my plate before my child is born.
So let’s see. What has been a difficulty of contributing?
The stuff that’s probably hardest is just dealing with people. The software side of things generally, is relatively easy. I mean even the stuff I don’t really want to do, I can just do it and it’s fine. But then you know, sometimes people are conflict adverse or conflict happy or rude or entitled or whatever it may be, and that kind of ends up being problematic. I guess in the past few years, like a lot of human relationships, the most troubling ones are when there’s actually a closeness there.
So although if get kind of abusive comments and it’s someone you’ve never talked to before, then that’s fine. Whereas we’ve kind of had conflicts between some maintainers and various allegations that get made and things like that. And like the lines start to blur a bit between. I guess it’s almost like in a workplace, where you have people who you work alongside, but then they’re asking you to kind of help out and intervene in their personal life and stuff like that, and you do that, but that ends up causing issues and this sort of thing.
You said you have governance policies, do those extend to this kind of thing.
Yeah so we have a code of conduct document as well which sort of defines the high level behaviour that is expected of people. That basically is what makes it - particularly with, random individuals you have no contact with - if their opening comment on the project is one that is racist or sexist or whatever it may be - then that makes it very easy to be like well we said this, you were pointed to this to get involved in the project, therefore you get blocked from the project or whatever it may be.
We do have guidelines for how maintainers shouldn’t talk to each other, but obviously those lines are a bit more blurry. Even culturally, when you work with people from the U.K. and the U.S. and Russia and China and places, what is considered respectful or rude or polite or helpful - typically when most of your communication is textual is quite broad and open to interpretation - it’s kind of tricky.
So it sounds like people have worked on easing the difficulty of when things come up in the social side of the project, and try to make it a good environment?
Yeah I think so. Yeah that’s fair.
What tools do you use to work on, or to contribute to Homebrew? And I’m using tools in a really broad sense.
Yeah, cool. So GitHub and the products that it’s hosted by. I use that pretty heavily. I use Git the version control system and I use Visual Studio Code. Currently it’s my text editor of choice and then I use a plethora of other open source tools to do bits and pieces here and there. Those initial three would be the main three things that I use.
You mentioned using open source tools. What is your relationship with open source, itself? Some treat it as political, some might treat it pragmatically in terms of development, etc. There are different perspectives on what it means. I’m just curious how you relate to open source?
Yeah, so for me it’s partly a pragmatic thing. Working as a professional software developer, it makes life easier working with open source software than with closed source software in general. Because when I have problems I can peak underneath the hood and make modifications and such. I guess the ideological part, I used to maybe be more into.
I still think that ultimately if I could choose between most software being open source or not… if I’m picking between a tool that I’m going to use, for example, my text editor, I prioritise using an open source, maybe slightly inferior text editor where a closed source may be a slightly better one. But it would not be completely overruled if there was one that was vastly better, than closed source or vastly inferior, whatever it may be. And yeah, I generally see - in terms of my personal contributions - I generally see open source as being a kind of very, minimal sort of charitable work. I don’t think it’s the same as volunteering at a homeless shelter or whatever. I feel like that’s a lot more worthy. But I feel like it’s not net negative and it’s probably very slightly above net-neutral for the world.
Particularly the project I work on. For it to be useful - well I guess we’ve opened up a little more - but historically, for it to be useful you need to be using a Mac, and that’s a subset of the world and a relatively privileged subset at that. I see it as being positive, but not overwhelmingly so in that respect. Open source in general, I feel, is a valuable, positive contribution that has levelled the playing field. Organisations, individuals and companies with different levels of money now don’t have vastly different tools, as a lot of them are now sharing the same playing field, in terms of their open source tooling.
How did you decide on which license to put on Homebrew?
So I’m not the original creator of the project. The original creator, - well, I’m not 100% sure why, but I could speak to other projects in which I’ve ended up using the same or a very similar license. Broadly I see there being a couple of main camps. You’ve got the copyleft camp of free software people, who generally want everything to be open source and they want to try and use their software as a means to accomplish that end, so they ensure their software is not used in closed source software. It forces closed source software to become free or open source software, in theory at least.
And then the permissive license camp, which I’m probably more at, which is the if you build something you don’t really care where it’s used. It’s more about it just being useful and you would like it to be able to be adopted by as many people and as may organisations as possible. So, I’m probably more in the latter camp for most things I build. But I choose on a project by project basis. Like, if a large company took this code and did something with it and made money off it, would I care or would I see that as being a good thing or a bad thing or whatever. And if I would see that as being a bad thing, I would maybe go for a copyleft license, and if I see that as being a good thing I would go for a more permissive license.
Okay. I want to go back to the pace of contribution. Have you ever been a regular contributor to a project, a open source project, but then stopped contributing? Could be Homebrew, could be something else.
Yeah I guess there have be a few other projects I worked on and stopped. And I guess there’s parts of Homebrew that I mostly don’t contribute to anymore.
Can you tell me about that? Why you might have stopped doing something and started doing something else? Or dropped an aspect of participation?
I guess it’s generally a factor of where I’m not really using the tool anymore. And what generally, pretty much all the open source projects that I ever contributed to, I’m using them at work or for some hobby project or whatever it may be. And I’ll generally contribute so long as that remains the case and I may get involved with the project for a while, while I do that. But then generally, when it comes to an end, for example might move jobs and I’m no longer using a particular technology anymore. Then my work on that will generally dry up as I don’t have the either desire, or some cases, the ability to contribute to that in an effective way.
Do you talk about aspects of retention in the project? I’ve heard that word thrown around in different spaces.
Yeah, we’ve talked about that a bit. So, my little concept that I’ve written and talked about is: I think that it’s like a kind of funnel. A bit likes a sales funnel, a kind of contributor funnel. With the idea being; every user is a potential contributor. Or every contributor is a potential maintainer. But then, for each level you go down the funnel, you get a fairly major kind of drop off.
So using that for retention and onboarding is sort of similar in that you’re trying to sort a bunch of people in the right directions and give them stuff to do. I guess in a retention case, for maintainers and contributors, if someone comes and just tries to solve some problems themselves, then it’s seeing if you can find other things that they could usefully get involved with and solve. I mean, ideally stuff which is going to be beneficial to them as well. But even if it’s not directly beneficial to them maybe you can find something that they find fun or interesting or whatever it may be, and that’s generally quite helpful in getting them to stick around.
That’s an interesting idea that any user is a potential maintainer. Where do you get that from?
I guess it was my own observation. And it’s just that I’ve been involved with Homebrew for about ten years now, and seeing more and more people get involved, you realise that people often need to be slightly nudged to get more involved. And again, when people become maintainers on a certain project, they’re generally always asked. It’s very, I mean I literally don’t think we’ve ever had someone who has explicitly said, “I would like to be a maintainer, how do I do that?”
It’s generally always people who - you have to ask them and often they will say, “no”, because they feel that they are not competent enough, experienced enough, whatever it may be, and you have to try and talk them around and be like, “no, you’re doing enough valuable work”. So that’s the side of it we’ve seen. And also, the user side, in terms of getting people to contribute, it’s often just a matter of making it as easy as possible for people to do what they want.
If stuff is well documented, if you have good tools, good processes in place, and if it’s easy enough for them to do that, then they can do. I guess our project is slightly different, compared to, say, Firefox, as not ever user is a potential contributor, because you have enough people who can’t write code, or people who just have less interest in that way using the project. Whereas the vast majority of Homebrew users are developers in some form.
Right.
Particularly when our bug tracker is on GitHub then it’s easier - if someone is signed up already for a GitHub account - to create an issue, then they have, already got most of the steps out of the way required for them to make a change and send that back to us.
You said when you might ask someone to be a maintainer if they were already doing a lot of work for the project. What does it mean to be a maintainer?
Yeah, so again it’s one of these things that we’ve written up a bunch of stuff around on our documentation site. Like we try and write out all the kind of guides to, like what is a maintainer, what are expectations, what are the expectations of contributors, etc. But for us, I guess the definition of a maintainer would be someone who is spending more time shepherding other people’s contributions than contributing themselves. Then, from a logistical perspective, a maintainer is someone who has access to the repository.
So when someone is submitting a change to the repository they need a maintainer to accept and merge the change for it to be included in the repository in the software. So, logistically, a big part of that job should be facilitating those changes. Because, if all you’re doing is making changes and getting someone else to merge your change, or whatever it may be, then arguably you’re not really adding much as a maintainer in that way.
Okay. That’s helpful. I wanted to ask how your work changes around release cycles. Is there different kind of tone to your work at that time? Is there a different community that happens there?
Yeah So we have fairly loose release cycles in that we do quite a lot of regular releases without any particular kind of timing or anything like that. And it’s generally just whatever is ready at the time - let’s just release it. So things don’t really change particularly from that perspective. When we do a more major release, if we have big changes that are kind of disruptive, we’ll try and clump them together at that time. I guess that’s probably all it is for us really.
Okay. So I guess I just wanted to ask a little bit more your experience in the project. I asked before have you ever stopped contributing to a space and to a project and you mentioned shifting your focus. Has there been any other projects that you started or stopped?
So there’s been none, I guess, started that have been still active that I’ve walked away from. There’s been some that I sort of started and a few people used them, but then they stopped being used by other people so then I walked away. A main example of one I guess I have been involved with in the past but then kind of walked away from is the KDE project, a desktop Linux environment. So I worked on that for a few years but then in their case I stopped using Linux. So that was the main thing was I just wasn’t really using it anymore. And I had been involved with it partly through some work stuff as well. But again even then, when I changed jobs then the work stuff I was doing completely stopped on top of that. So basically like no interaction between me and the project anymore.
Okay. I wanted to ask, do you feel like you get recognised for the work that you do on Homebrew?
Yeah, no, I mean I guess so. I mean in my local tech scene in Scotland, if I am speaking at a conference people may know in advance that I am the Homebrew person that’s in Edinburgh if they use Homebrew already. And from doing talks and all that people may have seen me speak about Homebrew and things. But that would be the main stuff. And then, when I’ve applied for jobs before, then if Homebrew is part of my resume then that has kind of helped me get jobs probably in the past.
So you said you get recognised from speaking as well. Is that something you’ve done much of?
Yeah, I do a reasonable amount of public speaking. Partly through work and then partly just through my involvement in the tech scene.
What do you think is the role of conferences in the community?
So I think they’re kind of a good way for people to get to meet other people who are working on similar things. So, for example, our Homebrew meeting we had where we did all those votes, we did that at a conference. Like the day afterwards so everyone could attend the conference and stuff like that, that was the FOSDEM conference in Belgium. So I think they are good ways of getting people together and I’ve worked from home for ten years as well, so it feels kind of similar to, in my job as well, when I feel like when people have met face to face and actually had social interaction they generally work better. They cut each other more slack and they’ve got more of a relationship in general and that’s a positive thing.
Do you have relationships with people in the project that extend past your collaboration on a more technical side of things?
Yeah. I guess not really. There’s people who I would consider borderline acquaintances, friends, whatever. We have a private slack and stuff like that, so when my kid’s born I’ll tell people and people will say congratulations and stuff like that. But there’s not anyone in the project who I would say I’m having difficulties with my marriage or a stressful time at work: what should I do? There’s no one who I have friendships with in that way.
Okay. I asked you if you feel you get recognised for the work you do. When you do good work on a project, who sees it or who knows?
I guess the main people who notice and know things are the other maintainers in the project. If I fix an issue for a particular user then they may well notice that and recognise that and say thanks or whatever it may be. But if it’s, say some new feature that’s added, a lot of people find makes things easier, then generally people don’t track the individual feature back to the individual who created it, to the same extent. Whereas the maintainers who are paying a bit more attention to the project may well be the ones who do notice that. If they’re reviewing pull requests and stuff like that. I guess they see changes attached to people’s names and therefore that may mean if they use the feature then they say, “Thank you - This will save me time.” Or whatever it may be.
How do you think that Git or GitHub affects what gets noticed or what work gets noticed?
I guess it’s that people don’t have a reason to interact with the GitHub project, generally, for positive reasons unless they’re submitting code. So generally people will create issues for problems they are having. Whereas generally people do not go, and - I’m the same with the projects I use, if someone adds some new feature and I think is amazing I generally don’t have the inclination to go and investigate who did that? Why did they do it? How do I say thank you? whatever. Occasionally I will do that if it’s something super dramatic where notice and it gets tweeted about and they say, “This person built this thing.” And it pops up and I see that then I’ll say, “Thanks,” then personally. But generally you don’t go digging to the same extent.
And when you use probably 40, 50 different pieces of software every single day, it’s kind of hard to keep track of what’s been done by whom and when and why and all that type of thing.
It’s interesting an interesting feature of interactions that people tend to thank less but when something goes wrong they notice that. What kinds of issues do you have to deal with in the project that others might not see?
I guess interpersonal issues between maintainers would be probably the main thing. So when you deal with stuff like that people maybe don’t see and don’t think about it. The continuous integration stuff I mentioned earlier, the testing machines, I think when you fix things that don’t have a GitHub audit trail, even if you were to go looking. So people are unaware that that work gets done.
And most of the stuff around softer skills, like making talks for the project, whatever it may be. Stuff that doesn’t end up packaged in software itself and doesn’t really get noticed to the same extent.
Okay. I was going to ask - what kinds of interactions happen on other platforms of communication like a mailing list? It sounds like you have a Slack?
Yeah. We do have some mailing lists, but they aren’t used very widely. They’re basically only used for things that have to be on email for whatever reason. So our communication between the team is either on GitHub, we try and do as much as possible as we can in public, and then stuff that either has to be private or is a bit more chatty or not relevant to people outside of the maintainers, so then that might be a message in Slack. So we might say the CI server seems to be broken or whatever, that might be a message in Slack. Or equally someone saying, “I’ve got a new job.” Or whatever it may be, little bits of chit chat as well.
And does anyone moderate those conversations or, what would you say is the sort of tone? Sounds like sort of a mix, there’s just really technical stuff and chit chat.
Yeah, so I would probably be the main person who’s doing the moderation. Moderation hasn’t felt like it’s been a problem for quite a while. We had people in the past who would speak to each other, both on Slack and on GitHub, including maintainers, who’d speak with unkind, sort of aggressive or passive aggressive language. But, that required a bit of nudging to convince people to stop doing that, which wasn’t always successful.
I mean now it seems to be pretty normal to just have pleasant, normal conversations. My goal is generally to try and have our interactions be like the workplace where things that you would consider appropriate to talk to a coworker about, you would consider appropriate to tell us or not and vice versa. Equally because we’re not all best friends. There’s stuff that you probably want to save for your conversations with your best friends and leave outside of Slack. And that goes to the code of conduct sort of stuff as well. If you have strong political opinions or whatever it may be, even if most of us maybe agree on certain things, it’s better to just keep that stuff separate, because you don’t know who’s going to be privately offended by things.
Right. It sounds like the culture has changed? Like the tone of the conversation has changed over time?
Yeah, I think so.
With the kinds of governance structures that have been in place?.
Yeah I think so. It’s sort of interesting because we, in our case, what predated me being an elected project lead was me appointing myself as the lead maintainer more or less because there was stuff that wasn’t getting done that I was doing pretty much all the kind of leadership of the project stuff myself. And I asked if they thought it would be reasonable for me to just appoint myself. And a few people said yes and some people said no. But then I just kind of decided to do it anyway.
And that ended up being, I think, the right thing to do, but it ended up obviously causing conflict. Because some people felt that there was a dictatorship or whatever, you could say. But I think that the tricky thing with open source is that it’s always a kind of balance because it’s not a company environment. You can tell people what they can’t do, but you can’t ever tell anyone what they should be doing, or must be doing or whatever it may be. Because you don’t dictate how people spend their time and you don’t have any way of “firing” people beyond just saying, “Well you can’t be a maintainer anymore.”
Do you think the word volunteer accurately gets what people do when they contribute to an open source?
Yeah I think so. I think, if they are volunteering and they aren’t doing that as a somewhat integral part of their employment, I feel like that’s a fair description. I guess there are people, even in Homebrew, for example, for whom their interactions with a project may be, at some times for, not in Homebrew’s case never all the time, but in a purely professional capacity.
So in those cases I wouldn’t class them as volunteers to the same extent when they’re doing that. But yeah I think in general that’s what people are doing. And in general you have the same, it feels like a similar obligation to like in person volunteering, in that no one can make you do things or not make you do things. But if you sign up for some volunteering thing at a soup kitchen or whatever, and then you just don’t turn up, then that’s seen as being a rude thing to do, regardless of the fact that no one can compel you to do it. But at the same time there’s some degree of social contract when you’ve agreed to do things.
I feel it’s somewhat similar with open source in that there’s an implicit agreement that you will clean up your own mess if you make a change and it caused a lot of issues for people then it’s probably mostly on you to try and sort that out. Or at least back out of your change or whatever it may be.
Framing it as a sort of social contract, it’s interesting. I asked what has been a difficulty this project. What do you think have been some of the rewards or the more nourishing aspects?
I guess the rewarding stuff is both intrinsic and extrinsic in that, I don’t think I would have probably got the job I have now had I not done any contribution to open source ever. And I like my job I have right now and I’m well compensated and it’s enjoyable. So that’s one aspect of it.
And more intrinsic stuff is - it’s a nice, fun hobby that is beneficial to other people at the same time as it’s enjoyable to me. And that’s a kind of nice double combination. The other thing is just that it’s a place where I’ve learned a lot of things and continue to learn. And I can experiment and there’s a degree of just satisfaction in the way that you can do development with open source. It’s, I would imagine, in some degree similar to the freedom that people describe as entrepreneurs, or whatever it may be. Where “no one can tell me what to do.” And “if I don’t like a particular way of doing something I can change it or I can justify why I think things should be changed,” and whatever it may be. So yeah that would be the main summation of what I have found valuable.
Where do you think that you’ll be with this project in say five or ten years?
Yeah, good question. I’m not sure. I would hope that in two years, I would hope that I’m still involved in a diminished capacity. So not even necessarily time diminishment maybe. And I would hope that in five or ten years that I’m not project lead anymore and there is someone else who is doing that job better than I could be doing it at the time. So I’m not sure. I’m not sure whether the project will continue to meaningfully exist in that period of time, stuff like that as well. It’s all quite hard to predict.
Where do you think the project will be as an infrastructure in terms of its use and how it looks and works?
Yeah in five or ten years… so because we’re primarily an Apple operating system tool, so much of what we do depends on the direction that Apple ends up going in. So Apple could release software that effectively ends up replacing us. And then I would probably abandon what I’m working on. Like Homebrew and trying work on the Apple thing or use that instead or whatever it may be.
But I think, I don’t know, I feel like the project will probably, in five years at least, assuming MacBooks remain similarly used for development as they do today, I think like Homebrew will continue to exist in some form and will still be used by people. I would imagine we will have figured out how to get rid of more of the bits of the project, like the testing stuff I mentioned earlier. Like figure out how to outsource that to other organisations who do it better than we can. And we are able to do more of the fun bits and less of the boring bits.
Has Apple ever directly reached out to Homebrew?
Yeah so we have some contacts with folks at Apple. It’s mainly on an individual basis and we don’t have like a big, official corporate relationship manager, anything like that. That would be very nice and useful. Because we often get slightly blindsided by a change they make, sometimes when the apps change we have to scramble around, and stuff like that. So we have a somewhat open communication channel, but not a widely used one.
This is probably my last question, which is a hypothetical question. What do you think would happen if tomorrow Homebrew was gone?
There are alternatives to Homebrew like MacPorts. So I reckon there would be a lot of grumbling and crying on the internet. But then, people would probably just migrate to a bunch would migrate to a tool like MacPorts, like another project that does solve similar problems in similar ways. And then a bunch of people would probably build their own equivalents or their own versions of something similar. And perhaps one of them would become popular enough to be widely used and perhaps not. I don’t know. Remains to be seen.
Okay. Well this was really helpful and interesting. I’ve used Homebrew myself and it’s really interesting to hear more behind the scenes aspect of it. Also its ubiquity and having a specific place in people’s workflows and that kind of thing… before contacting you I hadn’t thought a lot about who’s behind this.
Yeah, thanks!
]]>Your project should be as easy to contribute to as possible. There should be good, up-to-date documentation on what you can do to get started and how you can contribute to the project. Homebrew even has a single command (brew bump-formula-pr
) which allows you to make your first pull request without even using Git. Make use of pull request templates and automated checks (that contributors can run locally) to encourage high quality pull requests.
You can’t automate empathy, though. The most important use of your time on first-time contributors is congratulating them for being awesome when you merge their PRs so they contribute a second time.
So they came back for more? This meant that their first time was a pleasant experience: well done. The second-time contributor has taken a step closer to being a maintainer and as a result you should invest more time in them. Ensure that you start being a bit more strict with your code review. Don’t fix problems for them any more or tell them exactly what to do but instead make more ambitious requests of them. Some contributors will not like this; they do not want to learn from you or about why the project does things certain ways. If they do want to learn and are having fun doing so they may contribute a third time.
By the time someone has contributed more than two high-quality pull requests to your project they now warrant your individual, focused attention. You have a good chance to be able to mentor this person into becoming a regular contributor and maintainer. Suggest more issues they are likely to be interested in, praise their work and defend them against drive-by critiques.
These are the people you want to invest your time in rather than the first-timers. Do so and they’ll end up saving you more time than you invest in them.
]]>Firstly, let’s look at some good (and bad) reasons that organisations are now using OSS.
proprietary software costs 💵 and OSS is 🆓!
While this statement is technically true it’s grossly oversimplified.
With proprietary software:
In comparison, with OSS:
In fact, you may find for a given proprietary vs. OSS comparison the total cost of ownership is lower for the proprietary product (particularly if you need high quality support or training).
You may have heard someone on the internet (not me 🐧) say:
“Desktop Linux is only free if your time is worth nothing”
This alludes to the point I’ve made above. Desktop Linux works great for many people (not me, sadly) but it often requires more time to setup, configure and verify hardware compatability. When your time is less valuable (e.g. when I was a student) it’s more appealing than when your time is more valuable (e.g. when I’m working and have a family).
proprietary software costs 💵 and OSS is 🆓!
While this has elements of the truth, it’s a bad reason by itself to use open source.
the community will help us and 🛠 all our problems for 🆓!
There’s a fairly unfortunate and widespread assumption that when using OSS your training and support costs are reduced because you can contact the developers of the software directly. This leads to pretty unpleasant experiences; OSS maintainers end up massively overworked and corporate users are surprised that they aren’t treated like important customers.
It doesn’t matter if you’re a “big customer” of OSS (e.g. using it on thousands of your machines) like with a proprietary software vendor because you aren’t paying the OSS project proportionally to your usage (or, more likely: paying them anything).
Ruby on Rails has a great contribution guide 📚 that sums this up beautifully:
Then, don’t get your hopes up! Unless you have a “Code Red, Mission Critical, the World is Coming to an End” kind of bug, you’re creating this issue report in the hope that others with the same problem will be able to collaborate with you on solving it. Do not expect that the issue report will automatically see any activity or that others will jump to fix it. Creating an issue like this is mostly to help yourself start on the path of fixing the problem and for others to confirm it with an “I’m having this problem too” comment.
The benefit of OSS are you can collaborate together with others on solving your own problems and you have the access to the code and tooling required to do so.
the community will help us and 🛠 all our problems for 🆓!
In some cases with some software and some communities this may be true. However, if you’re a for-profit company relying on volunteers to fix your issues in a timely fashion, you’re being irresponsible towards your customers and a freeloader on the OSS community.
everyone else uses open source software now
Ten years ago this would be a contentious claim but it’s pretty hard to argue with now. Going from the earliest adopters of OSS to now we have:
This happened through the earliest to current widespread usage of different types of OSS:
everyone else uses open source software now 👍🏻
This is now both a true statement and a good reason to use OSS: it’s become the industry standard. Unfortunately, using open source effectively hasn’t yet become an industry standard.
GitHub has made it easy to quickly fork existing OSS to make your own modifications. The thought process one typically adopts when doing so is:
let’s just fork 🍴 and edit this for now
You have some change that your organisation needs to make, do so in your fork and run the fork on your servers. It works fine, you forget about it and so it becomes:
let’s just fork 🍴 and stay on our own fork forever
If you don’t have a plan on how you’re going to stop using the fork you’ve made, you’re accumulating technical debt; each future update to a new release will be painful and more time-consuming. Helpfully, many organisations ignore this by thinking:
updating to a new version might cause 🐛
Yeh, well, they might. However it’s also true to say:
getting hacked might cause 🐛
On top of any resulting bugs in your application your customers also aren’t going to be too happy with you if you end up losing or leaking their data. If you’re reliant on OSS at your organisation you should be able to answer the question:
how many libraries you use are vulnerable 🚨 right now?
Do you have tooling to answer this question on all of your internal software projects? I’m willing to bet for many organisations at least 50% of their internally forked OSS projects are running an outdated (or even a known vulnerable) version. Proprietary software obviously also has vulnerabilities but it’s not as trivial to have software to detect and address this (for good or for ill). If you’re looking for a solution for this for your organisation I’ve had great experiences on my OSS projects with Dependabot.
this issue is stopping me doing my job 👩💻!
Many OSS projects receive comments like this from Very Important Engineers (sometimes at some of the world’s largest software companies who should really know better). Thankfully, those folks are entitled to a refund:
A little cheeky, I know, but it’s worth remembering that, as I’ve written before, open source maintainers owe you nothing. They’ve provided you with software, tooling and a license to allow you to fix your own problems. If they fix your problems (lucky you!): great but you cannot expect someone you are not paying to do so for you.
How many times have you seen an internal project get open-sourced to an internet that doesn’t care?
let’s open source this so people will make it 👍🏻 for 🆓!
This is a common reason for making new open source projects as a company. Unfortunately in reality it’s more like:
let’s open source this so people will make it 👍🏻 if it’s already 👍🏻 and we help them!
The Open Source Contributor Funnel helps explain this a bit more; if you want to have lots of people contributing to or maintaining your recently open-sourced internal project you’re going to need lots of people using it first. That requires it to be useful, well documented and for a bunch of people to have heard of it. Homebrew (which I maintain) has millions of users, thousands of contributors and tens of maintainers today. Consider this when your internal project has ten external users and you’re wondering why none of them are contributing back.
The key to succeeding at using OSS is to participate in the community and let them help you (in the way they want to).
everything should be upstream 🏞
The upstream of a fork is the original OSS project you forked it from. This just means “get your changes back into the original project ASAP” which in turn means:
everything that can be should be someone else’s problem 🙅
Get those bug fixes and features on your outdated internal fork maintained by the upstream project and not you. This means you won’t need to manually apply security patches to your fork but can just upgrade like any other library. The process to forking a project well is:
If you follow these steps your experience using and modifying OSS will be much more pleasant. What if you don’t feel confident making changes?
help others help you by helping yourself ☝️
You need to be willing to put in the time and effort to make it easier for others to help you. Starting with the easiest:
Similarly on your issues, pull requests, tweets and everything related to open source:
If you want to read more about how to do all aspects of OSS well check out the Open Source Guides. These are the best single resource on the internet on how to contribute to, start and maintain an open source project.
Finally, just be a nice, kind human ☀️. It’s surprising how appreciated (and how rare) kind words are in OSS. Use them generously and you’ll reap the rewards.
]]>