Federation-Compatible Gateway Implementations

The following GraphQL Gateways support acting as an Apollo Federation Gateway.
Their support is tracked in The Guilds's gateway compatibility repository.

📖 Read more about our methodology and why we created this audit

Gateway Compatibility Test Cases Test Suites Full report
Hive Gateway 100.00% ✓ 170 ✓ 40 View report
Apollo Router 97.65% ✓ 166 ✗ 4 ✓ 38 ✗ 2 View report
Apollo Gateway 97.06% ✓ 165 ✗ 5 ✓ 37 ✗ 3 View report
Cosmo Router 74.12% ✓ 126 ✗ 44 ✓ 21 ✗ 19 View report
Grafbase Gateway 48.24% ✓ 82 ✗ 88 ✓ 16 ✗ 24 View report
✓ Passed tests
✗ Failed
Perfect compatibility
75% and higher
Less than 75%

Methodology and Purpose of This Audit

Advancing the GraphQL Ecosystem by introducing an open-source audit for Apollo Federation Gateways.

At The Guild, we are committed to advancing the GraphQL ecosystem through collaboration and open-sourcing knowledge and solutions. Recently, we decided to audit various GraphQL Gateways that support acting as an Apollo Federation Gateway. Unlike existing benchmarks that primarily focus on performance aspects, our audit measures the correctness and compliance of these gateways with the Apollo Federation specification. This audit is now fully open-source, and we are excited to share our methodology, findings, and detailed test results.

Why We Built the Benchmark

We built this audit for two main reasons. First, as maintainers of Hive Gateway (our Gateway for GraphQL Federation, previously known as GraphQL Mesh), in order to be fully compatible with the Apollo Federation specification, we needed a set of tests covering a wide range of aspects of Apollo Federation. Instead of building something solely for ourselves, we aimed to develop something beneficial for the entire community.

Second, we observed an opportunity to improve the testing and compliance of many GraphQL gateways with the Apollo Federation spec. By openly sharing our audit, we aim to help the entire community improve their implementations and achieve higher compatibility.

Audit Methodology

Our benchmark checks how well each gateway works with the Apollo Federation specification. We designed 160+ test cases, grouped into 40 test suites, to cover a wide range of scenarios. These test cases were crafted based on our experience with Apollo Federation and the feedback received from users of GraphQL Mesh (now Hive Gateway) and GraphQL Hive.

Audited GraphQL Gateways (in alphabetical order):

The testing process involves:

  • Setup: Each gateway is set up with the necessary configurations to support Apollo Federation.
  • Subgraphs: Each test suite contains a set of subgraphs that are started alongside a gateway. These subgraphs are GraphQL APIs serving data over HTTP.
  • Execution of Test Cases: The test cases are run sequentially against each gateway. These tests cover various aspects of the Apollo Federation specification, including error handling.
  • Result Analysis: After running the tests, we analyze the results to produce a compatibility score (percentage of the total number of test cases passed).

The audit is conducted in a controlled environment to ensure consistency and fairness. We run the tests on the latest version of each gateway, and we update the audit regularly to reflect changes and improvements.

The audit is fully open-source, and we encourage the community to contribute by adding new test cases, improving existing ones, or suggesting new features.

Our tests cover a comprehensive range of directives, including @interfaceObject, @key, @external, @provides, @requires, @extends, @inaccessible, @shareable, @composeDirective, and @override.

However, due to limitations of the Enterprise license of Apollo Router, we are currently unable to test the @authenticated, @policy, @requiresScopes, and @override(label:) directives. We plan to include these directives in our tests as soon as we have access to the Enterprise license.