Genii Weblog
Branching out: REST, gRPC, and other gory details you can (mostly) ignore
Mon 20 Jan 2020, 03:57 PM
Tweetby Ben Langhinrichs
Posts in this series:
- The tree you are busy hugging has new branches,
- Branching out: A few terms
- Branching out: REST, gRPC, and other gory details you can (mostly) ignore
- Branching out: No REST for the wicked - Part 1
- Branching out: No REST for the wicked - Part 2
- Branching out: Let's tackle Node.js
- Branching out: More CRUD with domino-db
- Branching out: Why the domino-db Document is not like NotesDocument
- Branching out: Models, msgs, and microservices
In this post, I'll dive briefly into a few gory details. While these are critical to me as an ISV, for many Notes/Domino developers, they are concepts you only need to understand roughly.
Traditional NRPC: Communication between tightly coupled client and server
In traditional Notes development, you have been spared most of the worry about how the Notes client and Domino server communicate. They just do. We don't think about how running a script or editing a document in a server database from your Notes client. It just works. But the way it works is the Notes remote procedure call, aka NRPC. To quote the documentation:
Domino® servers offer many different services. The foundation for communication between Notes® workstations and Domino servers or between two Domino servers is the Notes remote procedure call (NRPC) service.
Because the Notes client and Domino server are tightly coupled, we don't need to worry about how NRPC works. It just works. And since both the Notes client and Domino server share a lot of the same executable code, they can balance who does what and only communicate when absolutely necessary.
Problem: Communication between decoupled client and server
Opening up NRPC to other systems is problematic. It can be done via the APIs, but essentially requires that the Notes or Domino code is installed on that server, so you are really still having Notes code communicate with Domino code via NRPC. But what about when we'd like a web page on a non-Domino server to communicate with Domino? Or we want a Notes client or Domino server to communicate with SalesForce or other software run by other people not interested in installing Notes/Domino. We need other protocols.
HTTP - a transfer mechanism
Primarily, this means HTTP. Even a classic Domino web page communicating with Domino uses HTTP rather than NRPC, as does XPages. That is why you can sit in an Internet cafe in Brussels and work on a page served up by your Domino server in Montréal. You are using HTTP rather than NRPC. HTTP is stateless, meaning that every call is independent. A session may be maintained on the server, but every call has to prove itself part of the session. That can prove a heavy load.
Solutions chosen by HCL: REST and gRPC
REST - a protocol for web services
In 2000, Roy Fielding wrote his doctoral dissertation on REST (Representational State Transfer), a concept he defined. The idea was that companies could create a REST API and defined efficient and easy ways to interact with their system over the existing HTTP system. If they wanted other people to use their system, as many companies did, they made it a public REST API. Instead of submitting an entire web page and having the server process it, the developer could use web services to send commands and receive back information to act on. This allowed web services when enabled programming in a far more interactional way.
While there are variants, most REST APIs use JSON as the payload and HTTP as the delivery mechanism. There are many public unsecured REST APIs, but if authentication and security are needed, most use OAuth, a public open-source authentication system that can use your existing authentication with Facebook, Google, LinkedIn, etc. OAuth 2.0 is the current state-of-the-art, and Domino provides that for authentication.
In my last blog post, I mentioned Domino Access Services. That is the REST API that first IBM and now HCL have provided to allow accessing Domino data from a web service. Using a simple URL (which is usually a GET in HTTP), you can retrieve the contents of a Domino view or document as a JSON string which can then be parsed and used in various languages. Lots more can be done, but I'll talk about that in a different blog post. But for a simple example, here is a URL endpoint (as they are often called) to access the views and folders in a Domino database on my very local server:
and when I use that on my database that has Domino Access Services and has the database setting below
then I get a response in JSON with each view or folder (aka collection) defined. You'll notice it also gives the endpoint URL to get to the contents of each collection
gRPC - a transfer mechanism and protocol
While the REST API over HTTP was sufficient for most uses, some uses require greater efficiency. Since many of those uses involve Google, they created their own new-but-old take on remote procedure calls for highly efficient APIs. They realized that the worst part of RPCs was the stuff they were passing was so damn proprietary and included internal data structures and so forth, but JSON wasn't fast enough and XML was worse. So, they made the whole thing open source and used protocol buffers, and then defined an easy way to create and handle those protocol buffers in various languages.
When HCL started work on a Node JS project, they could have used a REST API, but decided it would be more scalable to go with gRPC. Mind you, you can install and use domino-db and never know it uses gRPC, but if you want a sense of how well it works, look in the dominodb/package/src directory:
What you find is JavaScript files. No C/C++/Java. Just simple JavaScript files that passes gRPC messages up to the Domino server where the Proton server addin is ready and waiting for messages. It handles all the nasty stuff, and passes back other messages. This means that you can use the Server and Database and Document classes on any lightweight platform that allows Node JS (and therefore, JavaScript).
The protocol buffers are defined in .proto files. Here is an example I built for one of our forthcoming products:
By comparison, that same method would have a JSON schema something like this if it were in a REST API:
Again, you may not need to know a ton about these, but the terms will be used. If you can't get enough of this stuff and want a more general discussion of the different web services technologies, start with Kristopher Sandoval's article, When to Use What: REST, GraphQL, Webhooks, & gRPC.
Copyright © 2020 Genii Software Ltd.
What has been said: