Learning tests are a great way to consume a REST API and validate use-cases before integrating it into an application.

I want to build an application that allows me to schedule posts for publication on LinkedIn. The first step is to understand if their API supports my use-case and how. In this post, I will show how I build a learning test suite for the LinkedIn API.

Learning tests are essentially unit tests that we build while learning the API. Learning tests can document our learnings in code. They can be executed and becomes a second source of documentation that is more targeted to the use-case. In the end, we will have a test suite that is capable of verifying our understanding. The verification part will come in handy every time the API has a new release. Even when the API team is careful, errors happen, and we want to know if anything that we depend on broke in a release. 

As an added benefit, we have a good foundation for the client we will build after we have verified the use-case. The learning test term was coined by Jim Newkirk, as mentioned in Clean Code. It is a more than 20 years old concept, so it has stood the test of time.

I have read much API documentation, and it usually fails in two areas. They don’t document the data fields’ format, and they don’t document how to handle errors. Usually, the focus is the happy path, but all edge cases and error paths are murky. In this respect the LinkedIn API has good documentation but in places it is a bit difficult to follow making verification essential.

APIs are interfaces to other codebases, and we are often not able to peek into the source code to learn how or why it works as it does. It makes documentation and our experimentations the only source of information.

With that in mind, I start my learning tests by validating the happy path. I want to verify that the use-case is supported and that I understand how to use the API. After that, I start poking at the edge cases to see what happens.

There are many benefits to creating a learning test suite before integrating an API into a system.

1. It allows us to validate that the API has the features that we assume it does.
2. We can document how the semantics are in edge cases and error situations.
3. It allows us to run the test suite regularly to validate that the API still works as expected. It’s essential since we are not in control of the release cycle of the API.
4. As we learn about the API, any mistakes are confined to the learning tests. We are more knowledgeable when starting the actual implementation.
5. It is faster to perform our experimentations as unit tests instead of experimenting through a larger system.

Development workflow

The workflow I use to create the learning tests is quite simple.

1. Start by reading the documentation - identifying a relevant endpoint that supports the use-case or part of it.
2. Find any dependencies on the endpoint. It could be authentication tokens, ids, or other required values.
3. Figure out how to retrieve the dependencies - write tests for those parts
4. Write a test to call the endpoint from (1)
5. Verify that the understanding is correct
6. Repeat until the whole use case is verified

Notice that I don’t write any production code. All the code is located in my learning tests. It conflicts a bit with the TDD teaching where we write the test and then write the production code to make it pass.

Learning tests are not restricted to testing REST APIs. They can be applied to any third-party dependency. In the LinkedIn example, I make HTTP calls directly. But it might as well have been through a client library. The result would be the same. 

I will only start writing the production code after I have a complete understanding of the API parts needed to support the use-case. Then I extract code from the tests into my production code. Continuously creating more tests and verifying that the existing tests don’t break.

LinkedIn API

It is quite an extensive API. It contains at least 25 areas with many endpoints in each area. The areas I’m looking into is the UGC Post API and Authorization. 

There is no need to create learning tests for the whole API, just the parts needed to support the use-case.

The purpose of the UGC Post API is to support sharing. So it should support the use-case of sharing a post.

The LinkedIn Consumer Solutions Platform enables sites and applications the power to enhance their sign-in experience using the world’s largest professional network. The Consumer Solutions Platform contains APIs to Sign In with LinkedIn and Share on LinkedIn.

LinkedIn docs

LinkedIn has no sandbox API

One of the biggest problems with the LinkedIn API is that there is no sandbox, only production. It isn’t a big problem for authentication, but if we create posts, they will be posted publicly on LinkedIn, which is not nice and blocks us from running the tests continuously.

Searching for contacts

When posting on LinkedIn, you can tag people using @ and typing their name. My thought was to build similar functionality into my application to mimic the LinkedIn editor as close as possible. But that option is not publicly available. It seems like LinkedIn have had issues with someone scraping data using the API. They have locked the functionality to trusted partners. It is still possible to search for organizations and 1.level contacts, which is an excellent initial feature.

It is limitations like this that learning an API will turn up. If we had started to integrate the API directly into an application we might have a hard time handling the change in understanding.

Publishing posts

Let’s apply the flow from above on the publish posts use-case.

1. The endpoint used to publish posts to LinkedIn is part of the UGC Post API, it is called https://api.linkedin.com/v2/ugcPosts
2. There are two dependencies, the OAuth access token and the author’s urn. 
3. Retrieving the dependencies are explained in the sections below.
4. You can see the test here. It is relatively small, four lines.
5. It took a few runs to get the JSON correct, but the test allowed me to see a post getting created on my public profile on LinkedIn. 

The initial version of the learning test is only to verify understanding. As soon as I feel confident enough that the use-case is supported, I start creating the production code. If we look at the code we see that the JSON string is hardcoded as a string in the test. See line 33

In a later version of the JSON generator has gone from a string to a real serializer with an object hierarchy. See line 51.

Interactive authentication in a test suite - getting the OAuth access token

LinkedIn uses an interactive OAuth2 flow for authentication, that is an issue from a test perspective because it is difficult to set up interactive tests. 

To make any request to the LinkedIn API, we need the access token. The token is tied to a real user account, so we need to have a user login using the interactive flow.

We usually don’t want interactive stuff in our tests because it makes them impossible to run automated. But since LinkedIn doesn’t provide a sandbox environment, we have to make due.

The purpose of the test Should_GetAndStoreLinkedInOauthAccessToken_When_UserLoginInteractively is to support the OAuth code flow and store the auth token.

It goes like this

1. Start a local webserver to receive the OAuth code
2. Open a browser on the LinkedIn auth URL
3. Exchange the code for an access token
4. Store the access token locally for further API requests

It is a highly complex test, and you can see the implementation here: 

Getting the access token is a prerequisite for all other tests in the suite, so it must run first. A smell that makes the tests depending on each other.

Getting the author URN

When posting on LinkedIn, a reference to the author is needed. It is possible to post on behalf of others, so the author’s reference is required.

With the Profile API, it is possible to request the profile data of the current user. As seen here

The profile data contains the user’s ID, which is easily convertible to a URN.

Getting started with your learning tests

Whenever you need to use a third-party API or library, you can use learning tests. 

The power of learning tests is that we separate our learning and experimentation from the production code. It is much easier to build an excellent initial integration with a solid understanding of the interface.

Apply the developer workflow to get a good start 

1. Start by reading the documentation - identifying a relevant endpoint that supports the use-case or part of it.
2. Find any dependencies on the endpoint. It could be authentication tokens, ids, or other required values.
3. Figure out how to retrieve the dependencies - write tests for those parts
4. Write a test to call the endpoint from (1)
5. Verify that the understanding is correct
6. Repeat until the whole use case is verified