14 API strategy gotchas to avoid
Good APIs are an essential part of good code. They’re the boundaries that developers draw when we split up projects, create libraries, and share our work. As Robert Frost said long ago, “Good fences make good neighbors.” If he were alive today, he would be writing this poem about APIs.
Increasingly, good APIs are also the foundation for a good business. They’re not just a gateway to some data and decision-making intelligence, but a product designed to monetize it. APIs track their users and figure out a fair way to bill them.
Alas, creating these APIs is not as simple as opening a TCP/IP port and waiting for the money to roll it. Unleashed this way, an API will take requests from every device on the internet, and all of the chaos, both malicious and innocent, will start knocking on the enterprise’s door. The challenge is to make an API easy-to-use and welcoming to the right developers while impenetrable to the wrong ones.
Here are 14 common API issues that can trip up your API strategy.
Overlooking the price of exfiltration
Many cloud providers bill for a long list of events and some are easy to forget. The cost of a machine per hour is obvious, but many forget that the bill can include “data exfiltration.” Alas, the main job of an API is to exfiltrate data to the users. Make sure to take this cost into account when pricing your API — and also when sketching out the architecture.
Ignoring the ‘data format tax’
Some data formats such as XML are not as efficient as others, and this can affect the size of the data packets that your API will return. One of my friends still likes to brag about how she cut the data bills for her company by more than 40% just by getting rid of extra tags and cruft in the data format. Keep the data format as lean as possible and keep the data focused on only the necessary bits to keep your bandwidth bills manageable.
Feature bloat
Sometimes developers like to be helpful and include extra bells and whistles. Most of the time, these don’t cause any problems, even if they aren’t used. But sometimes they leave security holes that go unnoticed.
The programmer who added the ability to execute arbitrary code to the Log4J library didn’t plan to create one of the worst security bugs in internet history, but when people forgot about the power of this seldom-used feature it became just that. In cases like this, it helps not to be too creative or clever. Sticking to the bare minimum is not always the best way to build a great product, but it can be a good way to create a secure API.
Forgetting to pre-filter
Most APIs don’t do much work themselves. They take the input and pass it along to some other code. One of the best services that an API can offer is pre-filtering to make sure that the inputs conform to expectations. Many of the worst security bugs came from attacks that abuse the naive goodwill of some API to overflow a buffer or generate an SQL injection attack.
Skimping on testing
Many developers have a few basic test URLs. If the right packet comes back from the API, they assume everything is running smoothly. In reality, the test results are often cached and the simple test URLs may only exercise the first layer. A good test suite will make sure to touch every part of an API, especially its connection with the database and any secondary APIs or services.
CORS misconfiguration
Cross-Origin Resource Sharing (CORS) issues can appear when an API’s response is mixed in directly with some other content in a browser. This can be a challenge for some API users who rush into using the API. Sometimes the answer is adding the tag Access-Control-Allow-Origin to the headers. Sometimes it’s better to build a full proxy into the stack.
Choosing the wrong authorization
Figuring out the right amount of authorization for your API is a bit of an art. Some data isn’t too sensitive. The API’s only job is to track users to figure out any bills. For these simpler cases, an unchanging random key may suffice. Other APIs, though, protect personal information that may be incredibly sensitive. For these cases, more secure protocols such as OAuth 2.0, OpenID, or JWT are better choices. There are already good libraries for both ends of the protocols so upgrading the security doesn’t require writing much new code.
Not looking for log file anomalies
When the software is running and there are no panicked phone calls from customers, programmers tend to take a nap. The log files for APIs, however, can help flag danger before it occurs. Are any particular parameters causing more trouble for the software? Does the load spike on particular days, at particular times, or when one customer shows up? Spending just a few minutes skimming the log files for bad response times can highlight where your hardware is underpowered or your software is glitching. Fixing those issues now will prevent a real meltdown later when some perfect storm delivers several nasty datagrams at once.
Caching incorrectly
APIs often want to speed up responses and save computation by caching the results for common questions. When caching works well, everyone gets an answer that’s still fresh enough for their purposes. But when caching is overly aggressive, users end up with stale answers and no way to know when they were generated. Using little or no caching will solve this problem, but at the cost of more computation and complexity. Finding the right balance can be a challenge.
Rolling your own vs. using a framework
APIs are now common enough that programmers have built good frameworks such as Swagger, OpenAPI, or any of the commercial versions from the cloud companies. You just write a few lines to specify the parameters and the framework does most of the work. These frameworks have good filters for stopping abuse and offer modern metering to track usage. Creating your own version is increasingly foolish.
Ignoring a free tier
If your API is meant for a general audience of developers from a wide range of companies, then a free tier can be the simplest way to attract new users. If coders need to ask the boss for a budget line and authorization, it’s harder for them to experiment and show everyone what your API can provide. The danger, though, is that the free tier will be so generous that these developers continue to use the API without becoming paying customers. Finding that right amount is challenging and API-specific. Some APIs are easy to cache because the underlying data doesn’t change and that means free users may be able to stretch access. Be stingy with data that’s easy to cache for long periods of time. But consider being more generous if the main output grows stale quickly or might never be cached.
Mispricing an API
The greatest challenge for an API manager who wants to sell access is finding the right way to price the data. Set the number too low and you miss your revenue targets. Set it too high and people walk away. All standard techniques for marketing and pricing are fair game. Some companies set aside premium tiers to capture revenue from big users. Others set a high public price and then offer generous private deals. There’s no one answer to the problem and people in business school write PhDs on these marketing challenges.
Not taking advantage of serverless
Tools such as Cloudflare Workers or AWS’s Lambda aren’t right for every API, but they can be ideal for many. They deliver a simple price point that’s almost directly proportional to usage making it much simpler to design a working business model. If the cloud company charges $X for each function invocation and each API call turns into one function invocation, well, it’s not hard to see that you’ll need to charge at least $X for each API just to break even on the infrastructure. All APIs have other costs and some may be substantial but at the very least, serverless models make it easier for bean counters to price out some of the variable costs for running an API.
Not having fun with error messages
Who first created the custom 404 message? There are several opportunities in each API to inject some humor. Empty or incorrectly formatted requests are obvious, but your particular API might have good places to hide Easter eggs. The engineering teams who design Siri and Alexa needed to have a response to “Open up the pod bay doors.” What will yours be?
