Which ActivityPub Server Requirements are Easy to Test?

by bengo aka @bengo@mastodon.social

Testing ActivityPub is a daunting task. There are dozens of implementations that don't always work together. The spec has three Conformance Specification Profiles and 50+ requirements refering to 3+ other specifications. Many requirements lack clear testable assertions or otherwise seem complicated to test. It's not always clear where to even begin.

Proposal

Focus on the 'easy to test' ActivityPub server requirements and improve their testability.

Rationale

Scope

Determining Difficulty of Server Testing Based on ActivityPub Server Resources

In this paper, we categorize the "Difficulty of Server Testing" of an ActivityPub Server Requirement based on the diversity and cardinality of ActivityPub Server Resources that are used in the requirement and/or a complete verification of the requirement. Generally these resources are either served by the ActivityPub Server test target or managed by the testing environment.

By ActivityPub Server Resources, I mean the various resources that an ActivityPub Server might serve. Often a single server hosts all these resources for an Actor, but it is also possible to consider servers that serve only one or a few of them, and there are some requirements that only describe some of these resources in order to test for conformance.

For example, a requirement that tests ActivityPub Server to Server Interactions may involve an ActivityPub Outbox served by the test target as well as an ActivityPub Actor and ActivityPub Inbox managed by the test environment.

ActivityPub Server Resources described by ActivityPub Requirements

These ActivityPub Server Resources are all parts of an ActivityPub Server:

There may be more resources worth describing, but these are the examples required for this illustration.

Difficulty of Server Testing Categorization

In order to bucket requirements in a way that, however crude, enables focusing on the 'easy to test' ones, we will use the following categories for "Ease of Server Testing" along with criteria sufficient to add a requirement to that category.

Requirements Categorized by Difficulty of Server Testing

Easy to Test ActivityPub Requirements

ActivityPub Object

ActivityPub Actor

ActivityPub Outbox

ActivityPub Inbox

ActivityPub Actor Collections

ActivityPub Object Collections

Medium Difficulty-of-testing ActivityPub Requirements

This will be added to over time to record categorization decisions, but has been omitted for now in order to focus on requirements that are easy to test.

Hard Difficulty-of-testing ActivityPub Requirements

This will be added to over time to record categorization decisions, but has been omitted for now in order to focus on requirements that are easy to test.

Conclusion

In this post, I've proposed:

Focus on the 'easy to test' ActivityPub server requirements and improve their testability.

I then proposed a method of categorizing the "Difficulty of Server Testing" for ActivityPub requirements by considering the ActivityPub Server Resources described by the requirement and needed in order to test the requirement in a Test Environment.

I applied this method and the result is a list of 11 requirements with Easy Difficulty of Server Testing (out of 50 total ActivityPub Requirements).

Some next steps for me are:

Some calls to action for you:

tags
  • blog