
As we continue to [move toward a stable specification](https://json-schema.org/blog/posts/future-of-json-schema), we have been analyzing the various components, behaviors, and features of JSON Schema to determine what can be included, what may need modification, and what (if anything) needs to be removed or replaced.  The primary question that we had was whether there were any planned changes that would cause problems for users with existing schemas, i.e. breaking changes.

To answer that, we needed user input.  So we posted across the internet in order to encourage users to participate in a [particular conversation on GitHub](https://github.com/orgs/json-schema-org/discussions/295) around how we treat any supposed breaking changes for the upcoming release.  The response was very much, "If you can help it, please don't break things."  But many people also said they would be less bothered if there was a defined migration path and tooling to help.

We also performed an internal survey of feature stability, involving all of the JSON Schema core team members.  We're pleased to report that the vast majority of Draft 2020-12 can be kept completely as-is.  There are a few keywords and behaviors that could use some tweaking, but mostly in ways that we feel are generally compatible with the current release.

Another common sentiment among both users and team members was that JSON Schema already has a history (and reputation) of including breaking changes when releasing new versions of the specification, meaning that a schema that validates properly with one version may not validate properly with the next.  This perception is one that we need to address.  To do that, **we need to orient ourselves and the specification so that we can promise schemas written to one version will validate consistently with every subsequent version**.

Unfortunately, in order to make that promise, there is one behavior that we absolutely must change in a breaking way: support for unknown keywords.

## What are unknown keywords?

A keyword is "known" if it is defined by a vocabulary listed in the schema's meta-schema (identified by the value in `$schema`).  By recognizing this vocabulary and continuing to process the schema, an implementation declares that it understands how to process all of the keywords that vocabulary defines.  Any keyword which is not defined by a vocabulary listed in the schema's meta-schema is considered "unknown."

All of the currently published versions of JSON Schema instruct implementations to (at a minimum) ignore keywords that are not recognized.  With the introduction of annotations in Draft 2019-09, implementations were given the option to collect the values of unknown keywords and report them to the user in the output.  This behavior was widely supported by the community because it meant that they could annotate and document their schemas without having to worry that their validations would be impacted.  For a schema like

```json
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "type": "object",
  "properties": {
    "firstName": {
      "type": "string",
      "database-field-id": 1234
    }
  }
}
```

a validator would either ignore `database-field-id` or include it and its value as an annotation in the output.  One can see how this feature would be extremely helpful.  However, this feature also blocks our ability to promise future compatibility.

## Why can't we still support this?

Suppose a user wrote the example schema above while using a validator written to conform to the 2023 version of the specification.  A couple years later, they want to upgrade the validator now that supports the 2025 specification.  However in that 2025 specification, we added `database-field-id` as a keyword, and its value is expected to be a string.  Suddenly the user's schema is no longer valid.  **We've broken a user by adding a new keyword.**

In order to prevent this scenario, we are forced to forbid keywords that are not declared by a listed vocabulary.  We recognize that this will break a lot of people up front.  However we feel that a promise of future-compatible specifications should take precedence over the immediate pain of having to change your schemas yet again (and potentially again perpetually into the future).

## What is being done to soften the blow?

We have an [open discussion](https://github.com/orgs/json-schema-org/discussions/329) on this very topic, and we invite you to weigh in.

Current proposals include things from simple prefixes on unknown keywords as a convention to indicate custom annotations to more complex solutions like inlined and ad-hoc vocabularies bundled into the schema that can define annotation-only keywords, like `title` and `readOnly`.  These options are effectively ways for the schema author to say, "I know these keywords aren't declared by any vocabulary, but I'd really like them in my schema.  Please disregard them."

## Will there be any other breaking changes in the next release?

Hopefully not.  We believe that removing support for unknown keywords will be the only change of any real negative impact, but that doesn't preclude the possibility of other changes that break things in less critical ways.

We recognize that making breaking changes like this is hard on developers, both users and implementors, and so we're doing our best to measure the changes that are needed and seek alternatives to breaks where possible.  We don't see this as an opportunity to go wild with changes.

In any case, we will be open and transparent about any changes that we make, breaking and otherwise.

## Sum up

It looks like we have to break some schemas with the next version.  However by doing so, we are able to promise that we won't in the future.

We have to break it in order to fix it.

_Cover photo by [Ken Suarez](https://unsplash.com/@kensuarez) on [Unsplash](https://unsplash.com/photos/4IxPVkFGJGI)_


Greg Dennis

The Last Breaking Change


[Last time](./the-last-breaking-change), I wrote about how we had to remove support for unknown keywords.  I ended that post with a note saying that we were still searching for a way that we can support non-functional custom keywords, that is, simple annotations.

In this post, I'd like to review the solution and how we came to it, as well as a brief overview of some of the other solutions we dreamed up and were offered.  Buckle up, 'cause it's gonna be a mild ride!

## What's the solution?

The `tl;dr` of it is that going forward, JSON Schema will treat any keyword that starts with `x-` as an annotation.

This solution was chosen for several reasons.

1. Ad-hoc keywords necessarily cannot have any functionality tied to them.  That is, they're only ever annotations; their values are just returned to the user or application without any processing by the schema.
2. Having ad-hoc annotations follow a convention makes them very easy to identify.
3. A prefix is a good convention to follow.
4. Reserving this prefix for ad-hoc annotations means that there can be no collision with keywords defined by vocabularies, now or in the future (preserving our compatibility promise).
5. Developers are generally familiar with the `x-` prefix as it's already commonly used in other arenas, such as HTTP headers, to denote custom data intended for recipients that understand it.

The one hesitancy we had for using `x-` was that its origins denoted experimental behavior.  However, it seems that in practice, this prefix used quite liberally for any custom data.  Since we're making custom data our expressed purpose, it seems like a good fit.

## Why a prefix and not some other solution?

After releasing the previous blog post and publicizing it as far across the internet as we could, we received feedback that custom keywords were used quite extensively and not supporting them would break users more severely than we wanted.  While we had already planned on still supporting annotative custom keywords, we didn't know what that would look like.  So, I started a [discussion](https://github.com/json-schema-org/community/discussions/57) with some of the loose ideas that the team already had.

There's a lot in that discussion and quite a few ideas, but I'll just go through some highlights here.  If you're interested in the details, feel free to read the discussion.

### Alternative #1 - Defining the prefix in a new keyword

This option actually builds on the one we selected by defining a new Core keyword (one that starts with `$`) that would contain the prefix used for that schema.  It's a fascinating concept that would allow schema authors to use the prefix they wanted.

It was pointed out, however, that in order for a schema to be validated by the meta-schema, the meta-schema would need to be able to read this new keyword to get the prefix so that it could ignore the keywords that started with it.  This requires a whole lot of new mechanisms that we don't currently have in JSON Schema, so it's not very practical at this time.

We also noted that we couldn't figure out what the scope of this keyword would be.  Would it be only the schema resource (indicated by `$id`) where the keyword was used?  Would it be the whole document?  What if we `$ref` to another schema resource or document that doesn't define a prefix?  There's a balance somewhere between inferring intent and requiring too much repetition. 

### Alternative #2 - Listing the custom keywords to ignore in a new keyword

This option defines a new Core keyword such as `$ignored` that would hold an array of the names of keywords to ignore.  This would allow schema authors to explicitly define the keywords they wanted to use.

Like alternative #1, this has the problem that JSON Schema doesn't currently have the mechanisms to perform the kind of meta-schema validation that would be required, as well as the same scoping issues.  It's also possible that a schema author could ignore a keyword that would later be added to the spec or some vocabulary, meaning that it _shouldn't_ be ignored, resulting in surprisingly wrong validations and violates our compatibility requirements.

### Alternative #3 - Inlined vocabularies that define keywords

This option allows for a vocabulary to be defined and described _within_ a meta-schema's `$vocabulary` keyword.  This is like alternative #2, except that the ad-hoc keywords are defined by a vocabulary so the normal JSON Schema process that enforces "no unknown keywords" wouldn't pick them up; they would be known.

We decided against this because it requires a lot of further development of vocabularies, a concept which is still under development anyway.  This solution would also skew that development toward solving this problem, which may not be the right direction for the vocabulary concept.

### Alternative #4 - A new keyword to contain all custom annotations

This option creates a new Core keyword such as `$extra` that simply houses all of the custom keywords that a schema author may want to use.

Undesirably, this creates a layer of separation between the annotations and the data they're trying to annotate.  But more importantly, a keyword can only create a single annotation (as it's defined now), and so a keyword like `$extra` would lump all of the annotations into a single large object rather than being more targeted like individual keywords would be.

### Alternative #5 - Support unknown keywords behind an option

Finally, this option just requires implementations to provide a configuration option to allow unknown keywords, defaulting to "disallow."  While allowing unknown keywords breaks our compatibility promises, a user explicitly setting this option is effectively acknowledging that risk.

This felt like [a step backwards](https://vlipsy.com/vlip/the-emperors-new-groove-this-is-a-step-backwards-0YryOW3W).  It just didn't feel like it aligned with the spirit of supportability we want for this project.

## How did we arrive at `x-` as the prefix of choice?

In the previous post, I linked to the discussion and invited everyone to support their preferred ideas or propose new ones.  We saw a lot more interaction than we have in the past, and that was great!

Additionally, we posted about the discussion on social media and we even found several posts (e.g. on reddit) that others made linking back to the blog post or the discussion (or both) and discussing on that platform.

Once we had determined that the prefix option looked like the favorite, Benjamin Granados, our Community Manager, created a survey that listed options for what that prefix could be.  It included the proposals we had in the discussion so far, `x-` and a handful of easily typeable symbols, as well as a "make your own" option.

We had 53 respondents, which may not seem like much, but it's more than we've ever had.  The results clearly showed that `x-` (with 17 votes) was the preferred prefix.  Tying for second were the "at" symbol `@` and the asterisk `*` with 12 votes each.

## Too long; Read anyway

Moving forward, prefix your custom annotation keywords with `x-`.

Another nice thing about this solution is that you don't have to wait for the next version of JSON Schema to come out.  You can start updating your schemas today.  `x-` keywords are compatible with all versions of JSON Schema that are currently published; they'll still just be collected as annotations.  And when the next version comes out, you'll already have migrated!

_Cover photo by [Mick Haupt](https://unsplash.com/@rocinante_11) on [Unsplash](https://unsplash.com/photos/dtTu8Ec_uAU)_


Custom Annotations Will Continue


Since Ben Hutton joined Postman to work full-time on JSON Schema [the past year](https://blog.postman.com/ben-hutton-joins-postman-to-lead-json-schema-strategy/), other top contributors like Jason Desrosiers, Julian Berman, and Greg Dennis have joined, and now all are full-time supporting JSON Schema. So today, I am excited to announce that I am joining as Developer Advocate with a focus on serving the Community, nurturing relationships with other organizations, launching programs to support and grow the Community, and increasing awareness through content, public speaking, and discussions in Twitter, Linkedin, Stackoverflow or Reddit.

## Who is this guy?

My whole career, APIs have been at the core of everything. In the first part of my career, I focused on the engineering aspect of building APIs, playing all the possible team roles. Later, I focused on API Design and Governance, leading API Programs, and most recently, my focus has been Developer Experience and Developer Relations because this human aspect is the most critical for a technology or product to be successful.

Having said that, I see myself as a mix of 1/3 Software Engineering 1/3 API Strategy 1/3 Developer Relations, but I am constantly learning and testing new things, and today I am investing in Product Strategy/Growth. With this diverse background, I hope to be a good complement to the current JSON Schema Team.

## But who is this guy for real?

I am a father of two who loves to spend time with my family and friends. I love mountaineering and team sports, and I think nothing is impossible if you truly believe it. I love this sentence by Nelson Mandela "It always seems impossible until it's done".

## Like to know more about me?

If you like to know more about me, check out [this intro post](https://www.twilio.com/blog/introducing-twilio-developer-evangelist-benjamin-granados) I wrote when I joined Twilio.

## Let's do this!

2023 will be a fantastic year for JSON Schema, and I am excited to be part of this journey with all of you! I can't wait to start connecting with you in the upcoming [working meetings](https://github.com/json-schema-org/community/discussions/35), in [Slack](https://json-schema.org/slack), and [Github](https://github.com/orgs/json-schema-org/discussions).

>* Email: benjamin.granados@postman.com
>* Twitter: [@benjagm](https://twitter.com/benjagm)
>* Linkedin: [benjagranados](https://www.linkedin.com/in/benjagranados/)
>* Github: [benjagm](https://github.com/benjagm)

Cover photo by [Jai Cano](https://www.jaicano.com/redes) on [Instagram](https://www.instagram.com/jaicano).

Benjamin Granados

Hello 2023, Hello JSON Schema Community


*This is the first in what will hopefully be an intermittent series on Bowtie.
I speak here without speaking on behalf of the JSON Schema organization in any official capacity -- Bowtie isn't a tool we The JSON Schema Team are "blessing" in some way today, though I have personal hopes it becomes a sort of official tool, and have developed it intending it to be owned by "the community" in whatever sense that means.
For the moment, I speak only on behalf of myself as its author :)*

The JSON Schema

```json
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "type": "string",
  "maxLength": 3
}
```

validates strings of length at most 3.
This behavior is what the specification says *should* happen.
Does every *implementation* of the specification, across the wide spectrum of programming languages, properly *follow* the specification?
What about for other more complicated schema / instance pairs?
This is the key question Bowtie tries to address -- how can we compare behavioral differences between JSON Schema implementations and between the specification, in the hopes that we make it easier to fix implementation issues and clarify any unclear portions of the specification?

## What It Is

[Bowtie](https://github.com/bowtie-json-schema/bowtie/) is what I'm calling a "meta-validator", by which I mean a program which can execute *other* JSON Schema validation implementations and collect results from them.

There's prior art in doing this sort of thing.
The [JSONPath Comparison project](https://cburgmer.github.io/json-path-comparison/) does this extremely well for [JSONPath](https://github.com/ietf-wg-jsonpath/draft-ietf-jsonpath-base), and [Nicolas Seriot's "Parsing JSON is a Minefield"](https://seriot.ch/projects/parsing_json.html) is a fantastic example for JSON itself.
Bowtie attempts to bring these ideas to JSON Schema.

From the existing [list of JSON Schema implementations](https://json-schema.org/implementations.html#validators), Bowtie already [supports](https://github.com/orgs/bowtie-json-schema/packages) *12* implementations across *9* programming languages, allowing anyone to run any of these implementations and see what they have to say about schemas and instances.

It ships with a [command line program](https://bowtie-json-schema.readthedocs.io/en/latest/cli/), but perhaps more excitingly, ongoing automated runs of this CLI have been set up, such that Bowtie emits a [report across all of its supported implementations](https://bowtie-json-schema.github.io/bowtie/draft2020).

To produce this report, Bowtie runs the [official JSON Schema Test Suite](https://github.com/json-schema-org/JSON-Schema-Test-Suite), which is our existing set of tests meant to exercise compliance with the JSON Schema specifications.
Many implementations already [use the suite within their own continuous integration](https://github.com/json-schema-org/JSON-Schema-Test-Suite#who-uses-the-test-suite), but this is the first time both users of JSON Schema as well as implementers can see results of running the suite in a single place, across many implementations.
The test suite already has great coverage of our specifications (specifically the validation portions of them).
The suite certainly isn't perfect, but Bowtie's results therefore cover the validation specification really well.

## Why It Might Be Interesting

The most noticeable thing Bowtie enables is an easy way to compare how close an implementation follows the specification.
This gives transparency to users of implementations, and also to the community about areas which might be hard to implement, or are commonly misimplemented.
Our hope is that this leads to improvement and energy to help fix issues, and overall to a stronger community!

Even beyond the test suite, Bowtie is capable of providing a uniform interface to the implementations it supports, meaning you can quickly access results from each one without needing to learn the language-specific API each implementation offers.
If you have a schema and instance and want to run it across many implementations, or a single implementation which you're not already familiar with or don't have set up, Bowtie can help.

## How Do I Run It?

If you're just interested in its output (i.e. reports on runs of the test suite), you can currently find them at the following links, corresponding to each version of the specification:

* [draft2020-12](https://bowtie-json-schema.github.io/bowtie/draft2020)
* [draft2019-09](https://bowtie-json-schema.github.io/bowtie/draft2019)
* [draft7](https://bowtie-json-schema.github.io/bowtie/draft7)
* [draft6](https://bowtie-json-schema.github.io/bowtie/draft6)
* [draft4](https://bowtie-json-schema.github.io/bowtie/draft4)
* [draft3](https://bowtie-json-schema.github.io/bowtie/draft3)

The medium-term goal is to combine these onto one unified report (at which point the links may change, though hopefully we'll put a redirect in place).

If you want to go beyond the test suite reports, you can run Bowtie locally as well, on whatever input you'd like.
Bowtie is written in Python and published [on PyPI](https://pypi.org/project/bowtie-json-schema/).
If you have no existing preferred setup for installing Python applications, [install `pipx` using the platform-specific instructions for your OS](https://pypa.github.io/pipx/#install-pipx), then run:

```sh
$ pipx install bowtie-json-schema
```

which should give you a working Bowtie, which you can check via:

```sh
$ bowtie --help
```

The usage instructions are in [Bowtie's documentation](https://bowtie-json-schema.readthedocs.io/), but more documentation is definitely needed, so do ping myself (Julian) if there are things you can't figure out, it will motivate documenting them.

## How Does It Work?

I won't go into full details of Bowtie's implementation in this post, but at a high level, Bowtie is a command-line program written in Python which orchestrates spinning up and down OCI containers ("Docker containers" in a loose sense).
The containers it runs are "test harnesses" -- little bridge programs which take an implementation of JSON Schema, written in any programming language, and then allow incoming requests from Bowtie to call into the library under test.
Adding support for an implementation essentially means implementing a test harness in this fashion (which is generally simple to do), and once the harness is present, the implementation will light up in Bowtie's reports, and any user of Bowtie will be able to execute the implementation via Bowtie's uniform CLI.

## How to Help

There are two areas you (a user of JSON Schema, author of an implementation, or community member) can help with:

### Improving Implementations

The first and most obvious is by helping to take any issues uncovered by Bowtie back to implementations (after confirming them), and then helping implementations fix them.

Find your favorite JSON Schema implementations and the tests it fails.
Are there existing open issues on the implementation's bug tracker about the failures?
If they aren't previously known, an issue ticket (with minimal reproducer) is likely going to be welcome by the implementer, so you may want to file one.
If they *are* known, you can file a pull request in Bowtie which contains information about the downstream issue, which Bowtie can make use of in reports (by showing that the test is explicitly skipped).
Here's an [example pull request](https://github.com/bowtie-json-schema/bowtie/pull/73) showing how to do so.

Look out for [Bowtie-generated badges](https://github.com/bowtie-json-schema/bowtie/issues/55) in the near future which you can use to show off your spec-compliance (nothing could be cooler, eh?).

### Improving Bowtie

The second is by helping to improve Bowtie itself, which I already have a long list of ideas for, which can only grow as more people start to use it.

To repeat part of the above, I myself have implemented support for [many implementations](https://github.com/bowtie-json-schema/bowtie/tree/main/implementations).
If you maintain or use one, look at the harness I wrote -- it's possible I misused your implementation (though so far thankfully hasn't happened).

I'll also mention that I've been experimenting with doing work on Bowtie on [livestreams here](https://www.twitch.tv/julianberman) -- without too much self-promotion, feel free to stop by and say hello, or even to flip through previous streams where I fight with various Bowtie-adjacent things, it may give you some pointers on how to work on it.

Below is some specific guidance around areas in need of help:

*This is current as of the publication of this post, but my hopes are that it swiftly becomes out of date. If it seems to be, feel free to reach out!*

#### Give Me Something Easy to Get My Feet Wet

There's a list of [good first issues](https://github.com/bowtie-json-schema/bowtie/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) on Bowtie's issue tracker which I've attempted to curate.
They cover various portions of Bowtie's codebase (the Python bits, the frontend bits, and its surrounding infrastructure).
The label doesn't indicate that the issue is necessarily "small" to fix, though many are.
But it does indicate the relevant functionality or fix is "straightforward" or likely well-scoped.
If you don't see activity on one, feel free to leave a comment if you begin to work on it.

#### I Want to Learn About a New Implementation or Programming Language

This is possibly the most "fun" thing to work on, or at least I quite enjoyed doing it.
Pick an implementation Bowtie doesn't already support from the [list of implementations](https://json-schema.org/implementations.html#validators) and follow the [tutorial for writing a harness](https://bowtie-json-schema.readthedocs.io/en/latest/implementers/) which will guide you through what Bowtie expects from a harness.
If there are gaps in the tutorial, do raise them or ping me!
But this can be a nice way to play with a simple program in a language you haven't used before, as well as a way to learn a JSON Schema implementation which may have made different choices than one you're already familiar with.

#### The Report is Ugly, I'm a Frontend Developer or UI Designer Who Can Help

This is probably the most beneficial area you can help in.
I myself am not a frontend developer, in case that wasn't obvious.
What I put together (with Bootstrap) is essentially the bare minimum of what's needed to show off results Bowtie emits.
If you have more experience or headspace to think about how to effectively report test results, or compare implementations, please help!
All of Bowtie's "frontend code" today lives in one place, [a Jinja2 template which gets formatted into a static single page site](https://github.com/bowtie-json-schema/bowtie/blob/main/bowtie/templates/report.html.j2).
Beyond generally "make it prettier, more responsive, or more usable", here are a few specific issues to look at:

* [Add client-side filtering and sorting](https://github.com/bowtie-json-schema/bowtie/issues/37)
* [Show combined errors+failures counts](https://github.com/bowtie-json-schema/bowtie/issues/53), or more generally improve the summary from a "table of counts" into a more graphically pleasing representation of what happened in a run
* [Display schemas & instances more prettily](https://github.com/bowtie-json-schema/bowtie/issues/28), or more generally, design a good widget or component for representing test cases, their schemas, instances, and (sub)tests
* [Show average compliance numbers](https://github.com/bowtie-json-schema/bowtie/issues/54), or more generally, what summary statistics *across* implementations should we show, and how should we show them?

#### I Develop Other Tooling in the JSON Schema Ecosystem

Great!
I have hopes Bowtie may hook in well with other upstream or downstream tooling.
As a concrete example, given that Bowtie provides uniform interfaces to downstream implementations, an "obvious" complimentary tool might be to fuzz-test across *all* implementations, looking for cases they disagree, or blow up, or more generally produce behavior non-compliant with the specification which isn't already covered by an explicit test in the suite.
Doing this likely simply means hooking such a tool up to Bowtie and letting it rip.
See [this issue](https://github.com/bowtie-json-schema/bowtie/issues/23) though it doesn't contain much beyond the above in the way of detail.

Other tools may also have nice interactive properties when combined with Bowtie, so if you have other ideas, reach out, or try it!

#### I Want to Contribute Tests

This is definitely also helpful.
If they're tests of the specification itself, check whether they're already present in the official suite, and if not, do submit them there.
If they're *not* tests that the official suite covers today, e.g. because they cover pathological cases like causing implementations to "blow up" rather than produce a result, they are now "fair game" to add (somewhere, TBD where), because Bowtie can "gently" run implementations and catch these sorts of errors.
Bowtie also allows for a wider range of `$ref`-related testing, because its protocol specifically instruments harnesses with a sort of "registry" of schemas that the implementation is expected to be able to reference.
[This issue](https://github.com/bowtie-json-schema/bowtie/issues/61) is the relevant one, but ideas are welcome here on kinds of tests we can now add.

#### I Found a Bug, or Have an Idea for Bowtie Itself
If it's with an implementation, you likely should take it to the implementation's issue tracker.
Please be polite, as there's still a small but non-zero chance that issues are caused by Bowtie itself.
And even if there isn't, please do be kind to maintainers, many (including myself!) are already aware of issues Bowtie is flagging and may really want to fix them but have things which make doing so difficult.
Help out implementations if you can!

If it's with Bowtie itself, or is a new idea related to Bowtie, definitely [file an issue](https://github.com/bowtie-json-schema/bowtie/issues), [start a discussion](https://github.com/bowtie-json-schema/bowtie/discussions), or start a thread on the JSON Schema Slack.

#### I Want to Do Something Random and Cool

There are some funky ideas in this bin, even already.
I would love to figure out whether we can make use of Bowtie to *benchmark* implementations uniformly, rather than just test them.
Being able to [jump into a language-specific REPL](https://github.com/bowtie-json-schema/bowtie/issues/38) is also a kind of out-there idea but one which might be fun to play with.
There likely are many others!

#### Setting Up Bowtie Was Hard But I Figured It Out / I Want to Help With CI, "Infrastructure" or Triaging

These are all under-appreciated areas you can help with, so please do reach out if you're interested in anything here.
Even something like making Bowtie easier to install for those who don't use Python is a nice help!
(Right now, builds produce a `shiv`, but not a cross-platform one.)

## Conclusion

Thanks for taking the time to hear a bit about Bowtie.
Special thanks also must be given to [Postman](https://www.postman.com/) who employs me full-time to be able to do work like this on behalf of the community.
Without Postman, this work would never have happened!
I hope there's a lot more to come.
Please do share feedback, it's very welcome, and if you do want to get involved, that'd be very much appreciated!

Cover photo generated via Stable Diffusion.


Julian Berman

Introducing: Bowtie


## Challenge

The Internet of Things (IoT) is the network of physical "smart" objects that
exchange data with other devices over the Internet. While communication and
interoperability are by definition the crux of the Internet of Things, the
emergence of custom or proprietary solutions results in devices that cannot
talk to each other due to differences in data interchange mechanisms.

To integrate these disparate devices, developers must work with a growing set
of protocols, serialization formats and API specifications. This results in
repetitive, non-scalable and error-prone work that is difficult to automate.

While technologies like OpenAPI and AsyncAPI largely solve this problem in the
context of Web APIs, they fall short for describing networks of non-HTTP and
multi-protocol devices, and do not consider different modes of interaction
based on their meaning in the physical world.

## Solution

To solve these problems, the W3C Web of Things works on providing standardized
building blocks that make use of JSON Schema.

JSON Schema is used for validating descriptions of network-facing capabilities
of physical entities called [Thing Descriptions][thing-description], and to
model and describe data sent by Internet of Things consumers and producers in a
multi-protocol manner.

The W3C Web of Things specifications are on board with JSON Schema since Draft
4, and even the first draft versions already catered to data models of Internet
of Things devices.  "Currently, we are on JSON Schema Draft 7 and expect to
move to JSON Schema
[2020-12](https://json-schema.org/specification-links.html#understanding-draft-names-and-numbers)
or a newer one when starting our new charter in February 2023," said Ege
Korkan, Research Scientist at Siemens and W3C Specifications editor.

"We observe JSON Schema becoming more and more stable over the years and
Postman's support to the community gives us even more confidence on it,"
continued Ege Korkan.

JSON Schema is delivering exciting features, and more importantly these
features are becoming more and more consistent across implementations.  For
example, Ege Korkan added, "JSON Schema vocabularies are very promising and we
plan to explore them further in conjunction with Semantic Web technologies."

## Impact

The W3C Web of Things relies on JSON-LD for linking vocabularies and ontologies
that already exist on the web, such as [SAREF][saref], [Units of
Measure][units-of-measure] and [Schema.org][schema-org]. Being based on the
JSON data model, JSON Schema fits well with JSON-LD when integrating both
technologies as part of a single specification.

Adopting JSON Schema means that the W3C Web of Things does not need to invest
effort on inventing another schema language. As a consequence of the popularity
of JSON Schema, existing tooling can be often reused instead of implementing
custom parsers, validators, code generators and UI generators for all the
programming languages that the community might be interested in.

JSON Schema has proven to be a foundational block for creating higher-level
tooling that is specific to the W3C Web of Things. As a notable example, the
community has developed a tool called
[testbench](https://github.com/tum-esi/testbench) that makes use of JSON Schema
definitions inside Thing Descriptions to generate matching payloads for the
purpose of stress and penetration testing.

Given the growing popularity of using JSON Schema to generate web-based forms,
W3C Web of Things specifications can be used to automatically generate
dashboards and user interfaces to interact with the objects modeled by Thing
Descriptions.

## Key Impact Results

The ultimate metric of success for a standard like the W3C Web of Things is
adoption, and community adoption is not something you can just "engineer".  Ege
Korkan commented, "since we adopt JSON Schema, developers who are familiar
with it have a smaller learning curve when learning the W3C Web of Things
standard."

"During my PhD times, my MSc engineering students had to become familiar with
the W3C Web of Things. These students typically took 1 week to go through my
[tutorial](https://github.com/tum-esi/wot-tutorial). If they knew JSON Schema,
that 1 week would become 2 days", said Ege Korkan.

## W3C Web of Things

The World Wide Web Consortium (W3C) is the standardization organization behind
most of the standards used in the Web. The W3C Web of Things Working and
Interest Groups work on the standardization of Web of Things at W3C.

The Web of Things groups aim to create an interoperability layer so that
different Internet of Things platforms, protocols and standards can operate
together.

The work started as a discussion in the Web of Things Community Group in
2013-2014.  It proceeded to become an Interest Group in 2015 that
collected the use cases and defined the standards to be worked on. Since 2016,
the working group is working on different standards on the Web of Things with
the first publications of the Thing Description and Architecture
recommendations in 2019 and more that will be published by the end of this
year.

At the time of this writing, the Working Group consists of 96 participants
representing 36 organizations and the Interest Group consists of 131
participants representing 48 organizations. 

![W3C Web of Things Examples](/img/posts/2022/w3c-wot-case-study/wot-examples.webp)

## Getting Started

The W3C Web of Things Working Group invites you to leverage your existing JSON
Schema knowledge to play with the Internet of Things through the use of the W3C
Web of Things specifications and tools. You might already have a smart device
next to you, just that it does not come with a Thing Description!

If you write your own Thing Description, you can programatically interact with
your device by using the [Node.js Web of Things
implementation](https://github.com/eclipse/thingweb.node-wot), or play with it
through the
[Node-RED](https://www.npmjs.com/package/node-red-contrib-web-of-things)
integration.

To learn more, head over to the
[documentation](https://www.w3.org/WoT/documentation/), watch the [introductory
videos](https://www.w3.org/WoT/videos/), have a look at the many examples
provided as part of the [Thing Description specification](thing-description)
and try them out on the [Thing Description
Playground](http://plugfest.thingweb.io/playground/) app.

Thank you to Ege Korkan, Web of Things Researcher at Siemens AG, Munich, the
W3C Web of Things Working and Interest Groups and Coralie Mercier from the W3C
Marketing and Communications Team for allowing and enabling us to share this
case study with you.

Cover image: [Hello WoT](https://bind.systems/tags/web-of-things/) © 2022 by desertmonitor OÜ is licensed under CC BY-ND 4.0

[thing-description]: https://www.w3.org/TR/2020/REC-wot-thing-description-20200409/
[saref]: https://saref.etsi.org
[units-of-measure]: https://bioportal.bioontology.org/ontologies/UO
[schema-org]: https://schema.org


Build more. Break less. Empower others.

JSON Schema enables the confident and reliable use of the JSON data format.

Used by

Why JSON Schema?

Streamline testing and validation

Exchange data seamlessly

Document your data

Vibrant tooling ecosystem

Start learning JSON Schema

Explore the JSON Schema Ecosystem

Welcome to the JSON Schema Community

Join the JSON Schema Community Slack!

Welcome to our blog!

The Last Breaking Change

JSON Schema Community Meetings & Events

Upcoming events¶

Contribute to the JSON Schema

Sponsors

Supported by