This blog post is about a number of hard lessons learned at the World Wide Web Consortium (W3C) while building new features for the Web with browser vendors and how political and frustrating the process can be.
The W3C Web Payments Community Group (a 220 person pre-standardization effort around payments on the Web) has been in operation now for a bit more than four years. This effort was instrumental in starting conversations around the payments initiative at W3C and has, over the years, incubated many ideas and pre-standards specifications around Web Payments.
When the Web Payments Working Group (a different, official standardization group at W3C) was formed, a decision was made to incubate specifications from two Community Groups and then merge them before the Web Payments Browser API First Public Working Draft was released in April 2016.
The first community group was composed of roughly 220 people from the general public (the Web Payments Community Group). The second community group was composed of a handful of employees from browser vendor companies (fewer than 7 active). When it came time to merge these two specifications, the browser vendors dug in their heels making no compromises and the Web Payments Community Group had no choice but to sacrifice their own specifications in the hope of making progress based on the Microsoft/Google specifications.
It is currently unclear how much the Web Payments Community Group or the Web Payments Working Group will be able to sway the browser vendors on the Web Payments Browser API specification. I do believe that there is still room for the Web Payments Community Group to influence the work, but we face a grueling uphill slog.
The rest of this post analyzes a particular W3C standards development anti-pattern that we discovered over the last few years and attempts to advise future groups at W3C so that they may avoid the trap in which we became ensnared.
A Brief History of Web Payments in and Around W3C
Full disclosure: I’m one of the founders and chairman of several W3C Community Groups (JSON-LD, Permanent Identifiers, Credentials, and Web Payments). A great deal of my company’s time, effort, and money (to the tune of hundreds of thousands of dollars) has gone into these groups in an effort to help them create the next generation Web. I am biased, so take this post with a grain of salt. That said, my bias is a well-informed one.
For a while now, the W3C Community Group process seemed to be working as designed. It took a great deal of effort to get a new activity started at W3C, but a small group of very motivated “outsiders” were able to kick-start the most supported initiative in W3C history. Here’s a brief history of payments in and around W3C:
- 1998-2001 – The W3C launches a Micropayments activity that ultimately fails.
- 2010 May – The PaySwarm open payments community launches outside of the W3C with a handful of people, a few specifications, and the goal of standardizing payments, offers of sale, digital receipts, and automated digital contracts on the Web.
- 2012 May – W3C staff contact, Doug Schepers, convinces a small handful of us to create the Web Payments Community Group at W3C and bring the work into W3C. In parallel, a W3C Headlights process identifies payments as an area to keep an eye on.
- 2012 – 2014 – Web Payments Community Group expands to 100 people with several experimental specifications under active development. Web Payments Community Group members engage the United Nation’s Internet Governance Forum, banking/finance industry, and technology industry to gather large organization support.
- 2013 October – W3C TPAC in Shenzen, China – W3C Management is convinced by the Web Payments Community Group and W3C staff contact Dave Raggett to hold a workshop for Web Payments.
- 2014 March – W3C Workshop for Web Payments is held in Paris, France. The result is a proposal to start official work at the W3C.
- 2014 October – First official Web Payments Activity, the Web Payments Interest Group, is launched to create a vision and strategy for Web Payments. It does so, along with the creation of a Web Payments Working Group charter. A great deal of the Web Payment Community Group’s input is taken into account.
- 2015 September – Second official Web Payments activity, the Web Payments Working Group, is launched to execute on the Web Payments Interest Group’s vision.
So far, so good. While it took twice as long to start a Working Group as we had hoped, we had far greater support going in than we imagined when we embarked on our journey in 2010. The Web Payments Interest Group was being receptive to input from the Web Payments Community Group. The Working Group was launched and that’s when the first set of issues between the Web Payments Community Group, Web Payments Interest Group, and the Web Payments Working Group started.
The Web Browser API Incubation Anti-Pattern
- 2015 October – A decision is made to incubate the Web Payments Working Group input specifications in separate Community Groups.
When W3C Working Groups are created, they are expected to produce technical specifications that expand the capabilities of the Web. Often input is taken from Community Group documents, existing specifications and technology created by corporate organizations, research organizations, and workshops. This input is then merged into one or more Working Group specifications under the control of the Working Group.
That is how it is supposed to work, anyway. In reality, there are many shenanigans that are employed by some of the largest organizations at W3C that get in the way of how things are supposed to work.
Here is one such story:
The Web Payments Community Group has a body of specification work going back 4+ years. When the Web Payments Working Group started, Microsoft and Google did not have a Web Payments specification. As a result, they put something together in a couple of weeks and incubated it at the Web Incubator Community Group. In an attempt to “play on the same field”, some of the editors of the Web Payments Community Group specifications moved that community’s specifications to the Web Incubator Community Group and incubated them there as well.
The explanation that the Microsoft and Google representatives used was that they were just figuring this stuff out and wanted a bit of time to incubate their ideas before merging them into the Web Payments Working Group. At the time I remember it sounding like a reasonable plan and was curious about the Web Incubator Community Group process; it couldn’t hurt to incubate the Web Payments Community Group specifications alongside the Microsoft/Google specification. I did raise the point that I thought we should merge as quickly as possible and the idea that we’d have “competing specifications” where one winner would be picked at the end was a very undesirable potential outcome.
The W3C staff contacts, chairs, and Microsoft/Google assured the group that we wouldn’t be picking a winner and would merge the best ideas from each set of specifications in time. I think everyone truly believed that would happen at the time.
- 2016 February – The months old Microsoft/Google specification is picked as the winner over the years old work that went into the Web Payments Community Group specification. Zero features from the Web Payments Community Group specification are merged with a suggestion to perform pull requests if the Web Payments Community Group would like modifications made to the Microsoft/Google specification.
Four Months of Warning Signs
The thing the Web Payments Working Group did not want to happen, the selection of one specification with absolutely zero content being merged in from the other specification, ended up happening. Let’s rewind a bit and analyze how this happened.
- 2015 October – The Web Payments Working Group has its first face-to-face meeting at W3C TPAC in Sapporo, Japan.
Warning Sign #1: Lack of Good-Faith Review
At the start of the Working Group, two sets of specification editors agreed to put together proposals for the Web Payments Working Group. The first set of specification editors consisted of members of the Web Payments Community Group. The second set of specification editors consisted of employees from Microsoft and Google. A mutual agreement was made for both sets of specification editors to review each other’s specifications and raise issues on them in the hope of eventually hitting a point where a merge was the logical next step.
The Web Payments Community Group‘s editors raised 24+ issues on the Microsoft/Google proposal and compared/contrasted the specifications in order to highlight the differences between both specifications. This took an enormous amount of effort and the work continued for weeks. The Microsoft/Google editors raised zero issues on the counter-proposals. To this day, it’s unclear if they actually read through the counter proposals in depth. No issues were raised at all by the Microsoft/Google editors on the Web Payments Community Group’s specifications.
Warning Sign #2: Same Problem, Two Factions
When the Web Payments Working Group agreed to work on their first specification, noting that there were going to be at least two proposals, the idea was raised by the Microsoft/Google representatives that they wanted to work on their specification in the Web Incubator Community Group. Even though the Web Payments Community Group editors chose to incubate their community’s specifications in the Web Incubator Community Group, the inevitable structure of that work forced the editor’s to split into two factions to make rapid progress on each set of specifications. This lead to a breakdown in the lines of communication between the editors with Microsoft and Google collaborating closely on their specification and the Web Payments Community Group editors focusing on trying to keep their specifications in sync with the Microsoft/Google specification without significant engagement from the Microsoft/Google editors.
This inevitably led to closed discussion, misunderstanding motives, backchannel discussions, and a variety of other things that are harmful to the standardization process. In hindsight, this was trouble waiting to happen. This sort of system pits the editors and the group against itself and the most likely outcome is a Web Payments Community Group vs. Microsoft/Google gambit. Systems often force a particular outcome and this system forces harmful competition and a winner take all outcome.
Warning Sign #3: One-Sided Compromise
A few weeks before the Web Payments Working Group face-to-face meeting, the editors of the Web Payments Community Group specifications discovered (through a backchannel discussion) that “the browser vendors are never going to adopt JSON-LD and the Web Payments Community Group specification will never gain traction as a result”. The Web Payments Community Group specifications used JSON-LD to meet the extensibility needs of Web Payments. This desire to not use JSON-LD by the browser vendors was never communicated through any public channel nor was it directly communicated by the browser vendors privately. This is particularly bad form in an open standardization initiative as it leaves the editors to guess at what sort of proposal would be attractive to the browser vendors.
Once again, the Web Payments Community Group editors (in an attempt to propose a workable specification to the browser vendors) reworked the Web Payments Community Group specifications to compromise and remove JSON-LD as the message extensibility mechanism, ensuring it was purely optional and that no browser would have to implement any part of it. We did this not because we thought it was the best solution for the Web, but in an effort to gain some traction with the browser vendors. This modification was met with no substantive response from the browser vendors.
Warning Sign #4: Limited Browser Vendor Cycles
Two weeks before the Web Payments Working Group face-to-face meeting, the Web Payments Community Group editors personally reached out to the Microsoft/Google editors to see why more collaboration wasn’t taking place. Both editors from Microsoft/Google noted that they were just trying to come up to speed, incubate their specifications, and make sure their proposal was solid before the face-to-face meeting. It became clear that they too could be stressed for time and did not have the bandwidth to review or participate in the counter-proposal discussions.
The Web Payments Community Group specification editors, in a further attempt to compromise, forked the Microsoft/Google specification for the purposes of demonstrating how some aspects of the Web Payments Community Group design could be achieved with the browser vendor’s specification and prepared a presentation to outline four compromises that could be made to get to a merged specification. The proposal received no substantive response or questions from the browser vendors.
Warning Sign #5: Insufficient Notice
Google put forward a registration proposal 12 hours before the Web Payments face-to-face meeting and announced it as a proposal during the face-to-face. This has happened again recently with the Microsoft/Google proposed Web Payments Method Identifiers specification. Instead of adding an issue marker to the specification to add features, like was being done for other pull requests, the editors unilaterally decided to merge one particular pull request that took the specification from three paragraphs to many pages overnight. This happened the day before a discussion about taking that specification to First Public Working Draft.
Clearly, less than 24 hours isn’t enough time to do a proper review before making a decision and if the Web Payments Community Group editors had attempted to do something of that nature it would have been heavily frowned upon.
Things Finally Fall Apart
- 2016 February – The Web Payments Working Group Face-to-Face Meeting in San Francisco
The Web Payments Community Group specification editors went into the meeting believing that there would be a number of hard compromises made during the meeting, but a single merged specification would come out as a result of the process. There were presentations by Microsoft/Google and by the Web Payments Community Group before the decision making process of what to merge started.
For most of the afternoon, discussion went back and forth with the browser vendors vaguely pushing back on suggestions made by the Web Payments Community Group specifications. Most of the arguments were kept high-level instead of engaging directly with specification text. Many of the newer Web Payments Working Group participants were either unprepared for the discussion or did not know how to engage in the discussion.
Further, solid research data seemed to have no effect on the position of the browser vendors. When research data covering multiple large studies on shopping cart abandonment was presented to demonstrate how the Microsoft/Google solution could actually make things like shopping cart abandonment worse, the response was that “the data doesn’t apply to us, because we (the browser vendors) haven’t tried to solve this particular problem”.
After several hours of frustrating discussion, one of the Chairs posed an interesting thought experiment. He asked that if the group flipped a coin and picked a specification, if there were any Web Payments Working Group members that could not live with the outcome. The lead editor of the Microsoft/Google specification said he could not support the outcome because he could not understand the Web Payments Community Group’s proposals. Keep in mind that one of these proposals was a fork of the Microsoft/Google proposal with roughly four design changes.
At that point, it became clear to me that all of our attempts to engage the browser vendors had been entirely unsuccessful. In the best case, they didn’t have the time to review our specifications in detail and thus didn’t have a position on them other than they wanted to use what was familiar, which was their proposal. Even if it was purely an issue of time, they made no effort to indicate they saw value in reviewing our specifications. In the worst case, they wanted to ensure that they had control over the specification and the Community Group’s technical design didn’t matter as much as them being put in charge of the specifications. The Working Group was scheduled to release a First Public Working Draft of the Web Payments Browser API a few weeks after the meeting, so there was a strong time pressure for the group to make a decision.
In the end, there were only two outcomes that I could see. The first was a weeks-to-months-long protracted fight which had a very high chance of dividing the group further and greatly reducing the momentum we had built. The second was to kill the Web Payments Community Group specifications, give the browser vendors what they wanted (control), and attempt to modify what we saw as the worst parts of the Microsoft/Google proposals to something workable to organizations that aren’t the browser vendors.
The Key Things That Went Wrong
After having some time to reflect on what happened during the meeting, here are the most prominent things that went wrong along the way:
- Prior to the start of the Working Group, we tried, multiple times, to get the browser vendors involved in the Web Payments Community Group. Every attempt failed due to their concerns over Intellectual Property and patents. Only the creation of the Working Group was able to bring about significant browser vendor involvement.
- The Microsoft/Google specification editors were not engaged in the Web Payments Interest Group and thus had very little context on the problems that we are trying to solve. This group discusses the future of payments on the Web at a much higher-level and provides the context for creating Working Groups to address particular problems. Without that context, the overarching goals may be lost.
- The Microsoft/Google specification editors were not engaged with the Community Group specifications. It was a mistake to never require them to at least review the Community Group specifications and raise issues on them. They were asked to review them and they agreed to, but the Working Group should have put a stop on other work until those reviews were completed. At a minimum, this may have caused some of the ideas to be incorporated into the Microsoft/Google specification once work resumed.
- The Working Group decided to incubate specifications outside of the Working Group, which meant that they had no control over how those specifications were incubated or the timeline around them. In hindsight, it is clear now that this was not just an experiment, but a dereliction of duty.
- The Working Group decided to incubate technical specifications between two groups that are grossly unmatched in power: the Web Payments Community Group and the browser vendors. Imagine if we were to throw members of the middle class into the same room as the largest financial companies in the world and ask them to figure out how to extend capital to individuals in the future. Each group would have a different idea on what would be the best approach, but only one of the groups needs to listen to the other.
Two Proposed Solutions to the W3C Browser Incubation Anti-Pattern
We know that the following patterns hold true at W3C:
- Attempting to propose a browser API with little to no browser vendor involvement from the beginning, regardless of the technical merit of the specification does not work.
- Attempting to propose a browser API where the browser vendors do not feel sufficiently in control of the specification does not work.
Giving browser vendors significant control over a browser API specification and providing feedback via pull requests is what currently works best at W3C. The downside here, of course, is that no one but the browser vendors can expect to have a significant voice in the creation of new browser APIs.
There are a few exceptions to the patterns listed above as some specification editors are more benevolent than others. Some put in more due diligence than others. Some standardization initiatives are less politically charged than others. Overall, there has to be a better way at doing specifications for browser APIs than what we have at W3C now.
The simplest solution to this problem is this:
As a best practice, a Working Group should never have multiple specifications that are still being incubated when it starts. The Working Group should pick either one specification before it starts or start with a blank specification and build from there. All discussion of that specification and decisions made about it should be done out in the open and with the group’s consent.
Barring what is said above, if the group decides to incubate multiple specifications (which is a terrible idea), here is a simple three-step proposal that could have been used to avoid the horror show that was the first couple of months of the Web Payments Working Group:
- Any Working Group that decides to incubate multiple specifications that do more or less the same thing, and will attempt to merge later, will always start out with one blank specification under the control of the Working Group.
- As editors of competing specifications finish up work on their incubated specifications, they make pull requests on the blank specification with concepts from their competing specification. The pull requests should be:
- Small in size, integrating one concept at a time
- Pulled in alternating between both specifications (with priorities on what to pull and when assigned by the originating editors)
- Commented on by all editors in a timely fashion (or merged automatically after 5 business days; it’s better to have conflicting material in the spec that needs resolution than to ignore the desires of some of the group)
- The Working Group Chairs merge pull requests in. If there is a conflict, both options are pulled into the specification and an issue marker is created to ensure that the WG discusses both options and eventually picks one or combines them together.
The proposal is designed to:
- Ensure that the Working Group is in control of the specification at all times.
- Ensure that proper due diligence is carried out by all editors during the merging process.
- Ensure that a mix of ideas are integrated into the specification.
- Ensure that steady progress can be made on a single specification.
While I doubt the browser vendors will find the proposal better than the current state of things, I don’t think they would find it so unpalatable that they wouldn’t join any group using this procedure. This approach does inject some level of fairness to those participants that are not browser vendors and forces due diligence into the process (from all sides). If the Web is supposed to be for everyone, our specification creation process for browser APIs needs to be available to more than just the largest browser vendors.