Powering Your API with Drupal 8

These notes accompany a presentation given at BioRaft Drupal Nights and at NYCcamp.

The image-heavy (but often context-light) presentation slides are on Prezi.

Drupal 8 comes with "Built-in Web Services", the official Drupal.org 8 promotion page is proud to say, but what can that mean for you as a developer (or someone who has some influence on Drupal developers)? The new feature teaser isn't too specific:

Build mobile apps with Drupal as the data source, or even post back to Drupal from the client. Drupal 8 implements state-of-the-art Hypertext Application Language (HAL) as well. Do you want to expose content as JSON or XML, authenticate a client with HTTP authentication, or expose views-generated lists as services? You can do all that and more. The possibilities are endless.

We will look at how Drupal's web services (thanks to the Web Services and Context Core Initiative, or WSCCI) makes it possible to use Drupal 8 to very easily create APIs that other sites, including other Drupal sites of different versions, can use to get information or provide information back.

And to that end we'll talk about REST, RESTful APIs (which are not as comfortable as they sound), and the HAL specification (which is not quite as scary as it sounds), and HATEOAS (which is not as hateful as it sounds).

Then we'll cover how to do this in Drupal 8 and demonstrate a practical working example for Drupal RESTful APIs which will put the web to perhaps the most reprehensible use a web developer can contribute to: Domain hoarding.

We will mostly not cover the really hard part, which is finding an existing definition for the resources. In REST, this means media types. See one e-commerce company's journey toward Hypermedia for how far down the rabbit whole this goes, and Toward HyperDrupal for the perspective of a wombat (Ryan Szrama of Drupal Commerce fame).

Accronym Hell (or, a Glossary)

REST, after all, comes from the people who brought us WWW for the World Wide Web— surely the only commonly-used acronym that has three times as many syllables as its full words.

API: Application Programming Interface
HAL: Hypertext Application Language
HATEOS: Hypermedia As The Engine Of Application State
HTTP: HyperText Transfer Protocol
JSON: JavaScript Object Notation
RDF: Resource Description Framework
RSS: Really Simple Syndication
REST: REpresentational State Transfer
SOAP: (word now used without original meaning, Simple Object Access Protocol, possibly because it's not simple)
XML: eXtensible Markup Language
WSCCI: Web Services and Context Core Initiative (OK, this one's all Drupal)

Your Presenter

Benjamin Melançon from Agaric, the international web technology company of three people. Hamburg (Stefan Freudenberg), Boston (Michele Metts), and wherever i'm wandering, usually Cambridge, Massachusetts, or New York. We are a collective, a worker cooperative where we are all workers and owners. And we are looking to grow by one.

Machine-readable information lets us do cool things

There are so many awesome things in Drupal 8, it's possible to lose sight of one of the driving forces of the massive code changes in Drupal 8: Web services.

Your site comunicating with other sites and machines out there. Even, the buzzword (buzzphrase?) that will be out of fashion soon i hope, the Internet of Things.

Let's take a moment first to review Drupal's strengths, the proven strengths in the past several, if not all, versions of Drupal— 4.7, 5, 6, 7.

1. Structured content
2. Displaying content on the web
3. Enabling many people with different roles, responsibilities, and access to create and engage
4. Adding ecommerce, publishing workflow, or other things other people have needed and put together a module or a suite of modules to do
5. Extending to your unique needs

Now we add an easy way to do web services, and the structured content isn't just for displaying on the web anymore but most anywhere you can imagine, the creation and interaction isn't restricted to a web site's interface, the functionality you can add can work with other machines on the Internet, and one of the things you can build is your own API with your precise needs.

What for?

• A mobile phone app retrieves recent news nodes
• On-site Javascript creates taxonomy terms from user input
• Deploying staging content entities to the production site
• A public REST API that allows clients to place commerce orders
• A private REST API to migrate third party data into Drupal
• Connecting Drupal to external systems
• Anything for which someone uses Kimono Labs on your site!

Mostly via http://lin-clark.com/d8-rest-slides/#16

RESTful API Modules in Drupal 8 core

• RESTful Web Services (REST module)
• Serialization module
• HAL module
• HTTP Basic Authentication module

https://drupal.org/node/783254

https://drupal.org/documentation/modules/rest

https://drupal.org/documentation/modules/serialization

Pretty awesome. So who do we thank?

• Lin Clark, linclark, quietly on gittip.
• Klaus Purer of epiqo, klausi on Drupal.org, welcomes support on gittip.

Why REST? The Web is based on REST

When you visit a web page, you are doing REST.

REST (REpresentational State Transfer)

An archetectual style for network services. A style or fashion. It's kind of like art, say, the impressionistic style. If you comply with a set of constraints, you are a part of that movement.

To borrow a metaphor from Roy Fielding (really), let us present Monty Python's architecture sketch.

That video sums it up, but if you prefer academic papers:

The important thing about an architectural style is that it encapsulates important decisions about the architectural elements and emphasizes important constraints on the elements of their relationships. The useful thing about a style is that we can use it both to constrain the architecture and to coordinate cooperating architects. Moreover, style embodies those decisions that suffer erosion and drift. An emphasis on style as a constraint on the architecture provides a visibility to certain aspects of the architecture so that violations of those aspects and insensitivity to them will be more obvious.

— From http://users.ece.utexas.edu/~perry/work/papers/swa-sen.pdf

The REST architectual style emphasizes that interactions between clients and services is enhanced by having a limited number of operations (verbs). Flexibility is provided by assigning resources (nouns) their own unique universal resource indicators (URIs). Because each verb has a specific meaning (GET, POST, PUT and DELETE), REST avoids ambiguity.

They developed the idea, but the thesis points out that it works. They gave some arguments for why those constrainst good, and that set of constraints is what they call REST.

Otherwise you just have HTTP to send serialized data, but that's not full REST. REST doesn't even imply HTTP, but all known REST implementations use HTTP. HTTP reflects the constraints of the Web REST

Easy way to log, that's the visibility constraint. Even more important is cacheing.

But you only have REST if you have links.

ATOM protocal is an implementation for posting information with XML. It has the concept of collections which are good for discovery, and gives you a link for how to add an item. When you add an item, it returns a result that tells you how to edit.

ATOM is REST. If they'd done ATOM in Drupal it would have a REST API, but Drupal 8 has a more convenient way to make a REST API.

The verbs are mainly for visibility, so when you look at request log you can see how many read and write requests are going on in your application.

Drupal 8 gives you lots of ways to do REST.

Just making an XML or JSON file... that's not making your view a service. A service is the whole thing, a whole application architecture.

Parts of the page

Drupal in the Post-page era

So it's a great idea to do Web Services as REST!

Hypermedia As The Engine Of Application State (HATEOAS)

Why do i even care what version your API is.

Remember, a RESTful API is not a REST system but only one side of it. Want to call yourself a network application specialist? Work both sides! If you are a back-end developer, you are half a network application developer.

One constraint of REST is code on demand: Send code from the server to the client to be executed there. You can take advantage of features the client has.

Table 5-1: REST Data Elements Data Element Modern Web Examples
resource the intended conceptual target of a hypertext reference
resource identifier URL, URN
representation HTML document, JPEG image
representation metadata media type, last-modified time
resource metadata source link, alternates, vary
control data if-modified-since, cache-control

article published with a permanent link, and also a link to the latest article.
Canonical URLs fixes this for SEO.

Creating an HTTP (hypertext transfer protocol) because they wanted

Development of HTTP was not an agile process. And that is good, becaue you cannot make a change every two or three months. Role of the network is to have a standard so that different vendors can communicate with each other.

REST is an architectural style.

HTTP is the protocol.

Your software can never be RESTful, in a way, because your software is not a network. RESTful API builds on top of a protocol like HTTP.

In REST you exchange data. Still with your request you can change application state of the server, so API in the broad sense. You can stick to the REST principals

From Roy Fielding's dissertation:

• a framework for understanding software architecture through architectural styles, including a consistent set of terminology for describing software architecture;
• a classification of architectural styles for network-based application software by the architectural properties they would induce when applied to the architecture for a distributed hypermedia system;
• REST, a novel architectural style for distributed hypermedia systems; and,
• application and evaluation of the REST architectural style in the design and deployment of the architecture for the modern World Wide Web.

Stefan: Backbone is the least RESTful thing you can do. It treats server as database... you are building in knowledge of the server into the client... just using HTTP as a way of serializing.

Not everything needs to be REST. Take a subset of the constraints, if you are interested in visibility for your server logs.

Don't really need hypermedia to have a client-side application you create talk to your server. You need it when letting other people's clients talk to your server.

The last mile is the hard part, developing standards for meanings of hypermedia links. Like HTML, an ongoing effort to define that language. HAL might be interesting. But a serialization format is

Links with meaning. That's what you need.

Stefan: Atom protocol is a really good example. What i did with my Drupal 7 Atom module is take the different content types as collections. Lets you fully control the state of the server.

Fielding calls the WWW an incarnation of a RESTful architecture. It's not one piece. It's the whole thing! What you are usually writing is not an incarnation of REST, but a piece of software that forms a part of that. But of course you can (and sometimes need) to use constraints in your server.

Stefan: People thought there would be specialized clients for e-commerce, that had an understanding of a shopping cart. [Didn't happen.] Most e-commerce is horrible in respect to these principles.

And cookies were never meant to be part of the internet. First thing people defining the Web thought of was how to scale horizontally: servers shouldn't have to know where you were before. (That way, you can keep adding servers and it doesn't matter which of the servers you get with any given request.) Nothing on the client side should tie you to a specific IP address, specific server. Cookies are a bad idea. Cookies are finally losing ground again. Basic HTTP auth built into Drupal 8 core. Authentication has always been the weak part of Web architecture.

Why should you know?

(Via Stefan)

If you are painting, you should know that you put oils on an oil canvas, and watercolors on other canvas.

You can ignore all that, but you will face consequences that may well get in the way of what you are trying to do.

There was a contender to REST for a while, map function calls to individual requests. Like a fine-grained API, treat the other side as part of your code. The Internet wasn't made for that. It was made for as few requests as possible. So a course-grained API works better.

It's painful to put the wrong paint on the wrong canvas. Or the wrong brushes on the wrong material.

Know about REST constraints to make your life easier. SOAP doesn't use HTTP verbs so you can't use server log to see what's going on.

Even if you aren't fully RESTful, don't work against the architecture of the Internet.

You can work against it but it costs. Session replication? Additional software? Less complex the better. (What Khalid always says on improving performance: Simplify.)

Drupal 8 doesn't just provide a RESTful API, it uses it too

• Backbone.js and Underscore.js are bundled into Drupal 8

Permissions, permisions

Gotcha: In addition to setting permission to do the HTTP verb on a given entity, you still need to provide the user role permission to create, view, edit, or delete that content type.

Instead of the extra permissions for HTTP verbs, Drupal should make an initiative to make more generic permission types that modules could use.

Serializers

XML support in core lacks attributes or namespaces.

Symfony serializer really nice. Creates normalized PHP array from any object, and then encodes it. The normalize step can be different or the same for different formats.

Change the output of a specific type of field by adding your own Normalizer. Yours is asked first, if it says it handles that type of feed.