Purpose

The purpose of this guide is to encourage consistency in the design and implementation of REST Services for campus consumption. However, experience has shown that opinions, preferences, and tastes differ among designers and developers; so this guide will avoid being overly prescriptive, allowing for the evolution of the standards as experience dictates. Peer review is a good practice and usually, leads to better overall solutions.

Disclaimer

This design guide is limited to the discussion of REST services. In addition, all the examples given below are examples only. The use of an example does not indicate that the resource discussed is currently available or will become available in the future.

Benefits of REST Style

REST provides 5 "constraints" that help guide toward a good API

1. Client–server - Separation of concerns is the principle behind the client-server constraints. By separating the user interface concerns from the data provider concerns, we improve the ability to create user interfaces across multiple platforms and improve scalability by simplifying the server components. Perhaps most significant to the Web, however, is that the separation allows the components to evolve independently, thus supporting the Internet-scale requirement of multiple organizational domains.
2. Stateless: Statelessness means that state is maintained on the client. This makes every request/response transaction independent. Stateless communications enable clients to recover from network errors, clients and servers can come and go without corrupting state, and new processing nodes can be attached without complex state management.
3. Cacheable - Cache control at multiple levels (client, server, intermediary) – because state exists in only one place. Caching at multiple levels decreases latency, bandwidth, and cost.
4. Uniform interface - This includes identifying resources, manipulation of resources, describing message types and application state transition. Resources are identified by URIs. These URI's serve as an abstraction and should not change often. HTTP verbs (e.g. GET, PUT DELETE) describe how to manipulate resources. Server state transition occurs only through actions that are dynamically identified within hypermedia by the server  Hypermedia protects the server from being locked into a URI scheme.
5. Layered system - Limits the complexity of implementing each layer and allows for multiple layers of security.

The Name, Context and Version are critical fields the uniquely identify an API.

They can not be changed and should be chosen very carefully.

At UCSD we are managing APIs with WSO2's API Manager and there are some additional requirements / problems that require special attention.  A known bug is that the UI is too permissive in what's allowed in these fields, which can cause errors when publishing or later.

Name

This is the "Display Name" of the API.  It is shown in the API Store and Publisher.  It should be the "Human Readable" name of your API

UCSD REQUREMENTS:

• Use UpperCamelCase
• Use only letters.  Numbers and underscores are allowed where necessary.
• Use a name that uniquely identifies your API within UCSD.
• Avoid long and short names.  2-3 words is common.
• DO NOT INCLUDE THE VERSION IN THE NAME
• Avoid including "API" in the name as it's redundant.

Context

This is the "url name" of the API.  It is what is included in the URL when calling the API.  It should be the "Machine Readable" name of your API. The Name and Context of the API should be almost identical, given one a user should be able to easily identify the other.

UCSD REQUREMENTS:

• Only a leading /, lowercase ascii letters, numbers and underscores.
• Take the Name, lowercase it and optionally add underscores.  Eg.  PreferredPronoun → /preferredpronoun or /preferred_pronoun
• DO NOT INCLUDE THE VERSION IN THE CONTEXT.
• The WSO2 API Manager supports several context formats; UCSD only recommends using the basic format (single leading /, no curly braces).  Invalid contexts cause a bug that causes the API to fail to deploy in which case it must be deleted and recreated.

Version

The WSO2 API manager treats multiple versions of an API as separate APIs.  Therefor the Version is used to identify which one a user is referencing. See below for more information on how Version should be used.

UCSD REQUREMENTS:

• Use numbers, dots (.) and an optional "v" prefix.
• The recommended format is a single number eg: "1".  See below for reasoning.

Example

• Name: PreferredPronoun
• Context: preferredpronoun
• Version: 1

There are four parts to the URI of a REST service: the host, context, API version, and the desired resource.

{host}/{context}/{api-version}/{resource}

Each part may be comprised of more granular elements and will be described in their respective sections.

• Each of the URL elements listed above MUST use snake_case

Host

The host will be in the following format:

The –qa indicator represents a non-production deployment.  When no indicator is present then the stage is production.

Context

The API context is used to determine which API an incoming request is attempting to access. The API Context MUST be unique and snake_case. When viewing APIs in the store and publisher the API Name is used. The API Name should be unique and human friendly. It is our recommendation that the context be a URL-friendly version of your API Name. Often we suggest removing special characters and setting it to lower case. Some long API Names can also be abbreviated. It should be clear which API Name and API Context is referencing, so when a developer is reading code and finds the URL that is invoked, they can find that API in the store.

API Version

The API version is used to determine which iteration of an API an incoming request is attempting to access. It is used in conjunction with the API context.  API version indicates breaking change in the interface (request or response) or functionality that prevents existing consumers from consuming this API successfully.

• Version only when needed. A new version of API also requires multiple runtime binaries and many times, backend database changes.

• Work with clients to see if they can be upgraded without increasing the version

• Support at the most 2 versions

• Version if the response has breaking change as well.

How to version:

There are different ways to version an API. Various techniques are based on indicating version number in:

• Headers

• URLs

• Headers & URLs both, using a Hybrid approach. Use the latest version by default unless a specific version is provided in the header or in URL

It is recommended to use URL based versioning as it specifically indicates what version is being used by the consumer. A version number should be appended to the host to allow for versioning of the resources.

Version Format:

It is recommended to format the version like: v1, v2, etc.  Optionally you can omit the “v” or include a minor version like: v1.2, 1.2, etc.  However, minor versions shouldn’t cause breaking changes so minor versioning in the url is discouraged.

Resource

The resource URL element specifies an entity in the current context.  A resource by itself represents a collection of entities. You can refer to a single entity by specifying a unique identifier.

General Conventions

• Resources MUST use snake_case

• Resources SHOULD be nouns and not verbs

• Resources SHOULD be plural

• Concrete resources are better than abstract

• e.g. /dogs better than /animal

Single Entity Resource

A specific resource entity can be targeted by adding the identifier to the URL.  For example:

The identifier MUST uniquely identify a specific resource entity.

Compound Resource Identifiers

If more than one identifier is needed to identify a resource, they MUST be concatenated with commas:

Qualified Resource Identifiers

When there are multiple types of resource identifiers, it is a good practice to qualify the identifier.

Sub Resources

Sub resources are used to interact with entities that can be uniquely identified by a higher level resource and have a direct relationship to the higher level resource.  For example:

This sub resource could be used to create a comment and associate it with an article.

Sub resources can also be identified with the same rules as top-level resources. For example:

Query Parameters

Query parameters can be used to specify the desired format of a response, for example: filtering, paging, sorting, etc.  Query parameters MUST be camelCase.

It is recommended to use the HAL specification to represent responses in json or xml.  HAL provides a set of conventions to express HATEOAS linking consistently.

• HAL Specification
• HAL Primer

Content negotiation

Content negotiation allows an API to provide multiple representations of a resource with the same URL.  Two headers primarily used to enable this are Accept and Content-Type.

• Accept - used by the client to specify which representation type is preferred in the response when there are multiple options available.
• Content-Type - used by both the client and API to specify the representation of the request / response bodies.

Single resource

Single entity resources are targeted with urls that end in a unique identifier.  For example:

Resources are composed of name / value pairs where the values can be basic data types, objects, or arrays.

Basic data types

Strings:

{
	"title" : "REST API Guidelines"
}

Numbers:

{
	"integer" : 1234,
	"negativeInteger" : -1234,
	"decimal" : 1234.01,
	"scientificNotation" : 1.2E+3
}

Boolean:

{
	"flag" : true
}

Date/Time:

Date and time formats MUST follow the ISO8601 format specified in RFC3339 Section 5.6.  Examples:

{
	"date" : "2019-03-25",
	"timeLocal" : "13:30:57",
	"timeUTC" : "13:30:57Z",
	"timePSTOffset" : "13:30:57-08:00",
	"dateTimeUTC" : "2019-03-25T13:30:57Z",
	"dateTimePSTOffset" : "2019-03-25T13:30:57-08:00" 
}

Objects

Resources can have attributes values represented as other objects.  If object is a sub-resource, meaning that can be specifically accessed via a URL, then it SHOULD be listed under the _embedded attribute following the HAL specification.  A self link should be provided as well for easy access to the sub-resource.

Arrays

Attributes values can also be arrays.  Arrays are collections of homogeneous values.  The values can be basic data types or objects.

Links

Links are used to provide possible actions that can be made on a resource given its current state.  More information can be found later in this document in the Links section.

Collections of resources

Resource collections are targeted with URLs that don’t specify a single resource with an identifier.  For example:

Even if a filtering option is provided that results in a single resource, an array of resources should still be expected by the user agent.  Collections of resources SHOULD be listed under the _embedded attribute, following the HAL specification, indicating that individual resources can be directly accessed via a self URL.

If the resources in the collection are large, then a reduced representation can be used that only contains the minimum attributes necessary for identification.  The reduced resources MUST be accompanied by a self link to retrieve the full resource by itself.

Paging

It is recommended to provide paging options on large collections of resources to let the user agent limit the number of resources retrieved at one time.  Additionally, links SHOULD be provided to facilitate the traversal of the paged collection.

More information can be found later in the document under the Pagination section.

Sorting

It is recommended to provide sorting options on large collections of resources.  Even if sorting options are not provided, it is recommended to specifically order the collection by default for consistent responses.

More information can be found later in the document under the Sorting section.

Filtering

It is recommended to provided filtering options on large collections of resources.

More information can be found later in the document under the Filtering section.

Request

The API MUST use proper HTTP methods, and operation idempotency* MUST be respected. Below is a list of methods that APIs SHOULD support. Not all resources will support all methods, but all resources using the methods below MUST conform to their usage.

GET /dogs

GET /dogs/10

POST /dogs

PUT /dogs/10

PATCH /dogs/10

DELETE /dogs/10

OPTIONS /dogs

*Idempotence is the idea that a client can make the same call repeatedly while producing the same result. In other words, making multiple identical requests has the same effect as making a single request.

**An operation is considered safe if the operation does not significantly alter the state of the application

Response Codes

HTTP status codes are used to categorize a response and MUST be implemented.  A description of the most commonly occurring status codes is listed below and a full list of status codes and their definitions can be found here.

Success Scenario

• 200 Default success response code when response data is required.
• 201 Response to a POST that results in a creation of a resource. Should be combined with a Location header pointing to the location of the new resource.
• 202 Operation deleted has not been committed yet.
• 204 Request successful but no content.
• 304 Not Modified - Used in response to conditional GET calls to reduce bandwidth usage. If used, must set the Date, Content-Location, ETag headers to what they would have been on a regular GET call. There must be no response body.

Client Error Scenario

• 400  The request is syntactically malformed.  Can apply to both request body and query parameters.
• 401  Unauthenticated. When no or invalid authentication details are provided. Also useful to trigger an auth popup if the API is used from a browser
• 403  The request is a legal request but user is unauthorized.  While none of the HTTP response codes match well with a business rule exception, use of the HTTP code 403 indicates that the server is refusing to fulfill the request.  This status code can be used when a business rule won’t allow fulfillment of the request - e.g. Enrollment when a class is full.
• 404  Not Found - When a non-existent resource is requested
• 405  When an HTTP method is being requested that isn't allowed for the authenticated user. 
• 409  Resource Conflict can be possible in complex systems. It can be caused by fulfilling the request. Duplicate entries, deleting root objects when cascade-delete not supported are a couple of examples.
• 410  Gone - Indicates that the resource at this endpoint is no longer available. Useful as a blanket response for old API versions
• 415  Unsupported Media Type - If incorrect content type was provided as part of the request
• 422  Unprocessable Entity - Used for validation errors
• 429  Too Many Requests - When a request is rejected due to rate limiting

Server Error Scenario

• 500 INTERNAL SERVER ERROR – The general catch-all error when the server-side throws an exception
• 501, Not Implemented. The server does not support the functionality required to fulfill the request. This is the appropriate response when the server does not recognize the request method and is not capable of supporting it for any resource
• 503 Service Unavailable. The server is currently unable to handle the request due to a temporary overloading or maintenance of the serve

Errors and Exceptions:

Errors should be as descriptive as possible. The following format is recommended

JSON payload

{
  "message": string,
  "developerMessage": string,
  "moreInformation": string,
  "errors": [
    {
      "message": string
    }
  ],
  "code": string
}

JSON payload (API Archetype example automatically generated upon an improper post)

{
    "errors": {
        "code": 400,
        "status": "BAD_REQUEST",
        "timestamp": {
            "year": 2019,
            "monthValue": 5,
            "month": "MAY",
            "dayOfMonth": 3,
            "dayOfYear": 123,
            "dayOfWeek": "FRIDAY",
            "hour": 23,
            "minute": 44,
            "second": 25,
            "nano": 132000000,
            "chronology": {
                "calendarType": "iso8601",
                "id": "ISO"
            }
        },
        "message": "Malformed JSON request",
        "debugMessage": "Required request body is missing: public org.springframework.http.ResponseEntity<?> edu.ucsd.its.dis.auditservice.controller.AuditController.postAudits(edu.ucsd.its.dis.auditservice.persistence.db1.domain.AuditWrapper,java.util.Map<java.lang.String, java.lang.String>,java.lang.String) throws java.lang.Exception",
        "subErrors": null
    }
}

Note:

• Either message or errors array MUST exist.
• All other elements MAY be used. If the value is null, the element SHOULD be omitted. An element SHOULD NOT be an empty string.
• Use an array of messages to capture multiple validation errors.

HATEOAS

HATEOAS stands for Hypermedia as the Engine of Application State and is a fundamental feature of REST APIs.  Basically it is the practice of providing relevant links associated with a resource or collection of resources to indicate the possible actions that can be made given the current state of the resource.

The primary function of implementing the HAL specification is to provide a consistent convention for HATEOAS links and it is recommended to follow this convention.  The HAL specification defines this structure for links:

_links : {
	"self" : {
		"href" : "https://api.ucsd.edu/context/v1/articles/1234"
	}
}

This link object can be included at the top level of any resource or _embedded resource.

Common single resource links

• self link - The self link provides the URL that points to the current resource or sub-resource.  You SHOULD provide this link on every resource and sub-resource in your API.  It's most useful in _embedded resources either listed in a collection or as sub-resources.
• state driving links - State driving links can vary based on your use cases and contexts.  They are a clear way to indicate specific actions that can be taken on a resource depending on it's current state.

Anchor
pagination pagination

Pagination

Paging support SHOULD be provided for large collections of resources.  Pagination is implemented by providing query parameters that indicate the number of resources to retrieve per page and where to start.  There are two recommended methods to provide paging:

• limit/offset - the limit indicates the page size and the offset indicates the index for a specific resource to start paging at given the current sorting and filtering provided.  The offset is a zero-based index.  This is similar to how many databases implement paging and because of this is usually easier to implement server-side.
• pageSize/pageNumber - this is similar to limit/offset except the pageNumber indicates an index for a set of resources given the current sorting and filtering provided.  The pageNumber is a one-based index.  This is probably a more intuitive way to think about paging from a paginated table in a web application.

If you're implementing paging on a collection of resources you SHOULD also provide pagination links to facilitate the traversal of the collection's pages.  Here is an example of the standard pagination links using pageSize/Number:

_links : {
	"self" : "https://api.ucsd.edu/context/v1/articles?pageSize=100&pageNumber=2",
	"next" : "https://api.ucsd.edu/context/v1/articles?pageSize=100&pageNumber=3",
	"prev" : "https://api.ucsd.edu/context/v1/articles?pageSize=100&pageNumber=1",
	"first" : "https://api.ucsd.edu/context/v1/articles?pageSize=100&pageNumber=1",
	"last" : "https://api.ucsd.edu/context/v1/articles?pageSize=100&pageNumber=13"
}

It is also recommended to provide metadata for your collection to indicate the total count or optionally the current page state of the collection.

An API SHOULD allow for the consumer to specify the order that the entities in a collection are returned.  The recommended method for sorting uses a query parameter called sort that can take a comma-delimited list of attributes to sort by.  Each attribute listed can be prepended with a minus sign '-' for descending or a plus sign '+' (or nothing) to indicate ascending. For example:

If the consumer does not specify a sorting preference the API SHOULD provide a default sorting definition.

It is recommended to provide filtering options on large collections of resources where it's appropriate.  There are several recommended ways to provide filtering given your specific use case:

Basic filtering

Recommended when only partial filtering support is required and filtering by equality is sufficient.

+ Simple to implement

+ Simple to use

- Can result in a long server-side parameter list to support large sets of filtering options

- Only equality filtering supported

Basic filtering with operators

A superset of the basic filtering option listed above.  Recommended when only partial filtering support is required, but complex filtering options are required.

+ Simple to use

+ Supports complex filtering options

- Can result in long server-side parameters lists to support large sets of filtering options

Advanced filtering

Allows the most dynamic filtering without long parameter lists server-side.  Recommended when comprehensive filtering is required on large sets of attributes.

+ Most comprehensive solution

+ Fewer server-side parameters

- Complex implementation

It is recommended to follow all of OWASP’s REST security guidelines and to specifically pay careful attention to:

• Always use SSL. No exceptions
• Keep it stateless as much as possible
• Avoid sequential numbers for the resource ID