Genii Weblog


Civility in critiquing the ideas of others is no vice. Rudeness in defending your own ideas is no virtue.


Tue 29 Dec 2020, 06:06 PM
Inline JPEG image
 
Exciton Boost 4.6.0 was released on our website today. You can find the release notes online.
 
This is a low key announcement because there probably aren't many reading blog posts about software this close to the New Year, but if you want to get a jump on the competition, you can request an evaluation license today and give it a try.

Copyright © 2020 Genii Software Ltd.

Wed 23 Dec 2020, 02:00 PM
Inline JPEG image
 
This is intended as a direct reference companion to my post earlier today, Getting data from Domino with Exciton Boost. Here, developers can see the exact requests made with parameters, HTTP headers, etc. in case anything in the video went by too quickly.
 
There are the nine tasks, but in some cases, I show multiple separate requests, as I used more than one in the video to explain different elements. For each request, I show the HTTP method and then one of different ways (e.g., JavaScript's fetch) to show how this code could be called. The second part is merely to get across the wide range of ways you could call Exciton Boost using different technologies. Finally, I include the video again in case you want to see these requests in action and see the results they generate.
 
Intro: Get core services
Returns a JSON array of objects, one for each view that is included, implicitly (all views) or explicitly (specified views), in the Exciton configuration db and marked as discoverable. Exciton is strict about data security, so views may be accessible but not discoverable. Views not included are inaccessible and will return a 404 error.
 
Intro) HTTP

GET /CurbAppeal.nsf/api/boost HTTP/1.1

Intro) cURL

curl --location --request GET '
https://geniisupport.com/CurbAppeal.nsf/api/boost'
 
 
Task 1: Get list of accessible views
Returns a JSON array of objects, one for each view that is included, implicitly (all views) or explicitly (specified views), in the Exciton configuration db and marked as discoverable. Exciton is strict about data security, so views may be accessible but not discoverable. Views not included are inaccessible and will return a 404 error.
 
1) HTTP
 
GET /CurbAppeal.nsf/api/boost/views HTTP/1.1
Accept: application/json
 
1) JavaScript - jQuery
 
var settings = {
  "url": "
https://geniisupport.com/CurbAppeal.nsf/api/boost/views",
  "method": "GET",
  "timeout": 0,
  "headers": {
    "Accept": "application/json"
  },
};

$.ajax(settings).done(function (response) {
  console.log(response);
});
 
 
Task 2: Get view with columns & metadata
Returns a JSON array of objects, one for each document in the view. Each document includes a link to get the document from the documents collection as well as item values for each view column. Note that there are parameters to page through the view if desired.
 
2) HTTP
 
GET /CurbAppeal.nsf/api/boost/views/76CA31CBCB19C70A8525694F0045C657 HTTP/1.1

2) Python - Request
 
import requests

payload={}
headers = {}

response = requests.request("GET", url, headers=headers, data=payload)

print(response.text)

 
Task 3: Get filtered view
Returns a JSON array of objects, a subset of the full view filtered by a key match with the first view column. There are alternative ways to filter the view, but this is easiest.
 
3) HTTP
 
GET /CurbAppeal.nsf/api/boost/views/76CA31CBCB19C70A8525694F0045C657 HTTP/1.1
X-Key-Value: Chagrin Falls, OH

3) JavaScript - Fetch
 
var myHeaders = new Headers();
myHeaders.append("X-Key-Value", "Beachwood, OH");

var requestOptions = {
  method: 'GET',
  headers: myHeaders,
  redirect: 'follow'
};

fetch("
https://geniisupport.com/CurbAppeal.nsf/api/boost/views/76CA31CBCB19C70A8525694F0045C657", requestOptions)
  .then(response => response.text())
  .then(result => console.log(result))
  .catch(error => console.log('error', error));

 
Task 4: Get all items from the document
Returns a JSON object containing each item found on the document, or at least all allowed. Items may be explicitly allowed or disallowed in the configuration database, or all may be allowed by default. Some items such as time/date, author and reader fields, and rich text items will be represented by subobjects. Rich text fields are rendered to high fidelity roundtrippable HTML.
 
4) HTTP
 
GET /CurbAppeal.nsf/api/boost/documents/524C08BF648BE77A85257B8600798F5B HTTP/1.1
 
4) Node JS - Request
 
var request = require('request');
var options = {
  'method': 'GET',
  'url': '
https://geniisupport.com/CurbAppeal.nsf/api/boost/documents/524C08BF648BE77A85257B8600798F5B',
  'headers': {
  }
};
request(options, function (error, response) {
  if (error) throw new Error(error);
  console.log(response.body);
});

 
Task 5: Get only selected items
Using an item list via URL or HTTP header, the request determines which items to returns in the JSON object. In addition to actual items, the request may use Notes formula language to create additional items or may use filtering to get subsets of rich text items.
 
5a) HTTP
 
GET /CurbAppeal.nsf/api/boost/documents/524C08BF648BE77A85257B8600798F5B?metadata=false&items=StreetAddress,Bdrms,Full=Bathrooms,Half=Half_Baths,Photo HTTP/1.1
 
5a) JavaScript - XHR

var xhr = new XMLHttpRequest();
xhr.withCredentials = true;

xhr.addEventListener("readystatechange", function() {
  if(this.readyState === 4) {
    console.log(this.responseText);
  }
});

xhr.open("GET", "
https://geniisupport.com/CurbAppeal.nsf/api/boost/documents/524C08BF648BE77A85257B8600798F5B?metadata=false&items=StreetAddress,Bdrms,Full=Bathrooms,Half=Half_Baths,Photo");

xhr.send();

 
5b) HTTP
 
GET /CurbAppeal.nsf/api/boost/documents/524C08BF648BE77A85257B8600798F5B?metadata=false HTTP/1.1
X-Items-List: StreetAddress+", "+City+", "+State=Address,Bdrms,Full=Bathrooms,Half=Half-Baths,Photo[Graphic 1]
 
5b) JavaScript - XHR
 
var xhr = new XMLHttpRequest();
xhr.withCredentials = true;

xhr.addEventListener("readystatechange", function() {
  if(this.readyState === 4) {
    console.log(this.responseText);
  }
});

xhr.open("GET", "
https://geniisupport.com/CurbAppeal.nsf/api/boost/documents/524C08BF648BE77A85257B8600798F5B?metadata=false");
xhr.setRequestHeader("X-Items-List", "StreetAddress+\", \"+City+\", \"+State=Address,Bdrms,Full=Bathrooms,Half=Half-Baths,Photo[Graphic 1]");

xhr.send();

 
Task 6: Add selected items to view results
Going back to the view, the request can include either additional or replacement items for the view columns using an item list via URL or HTTP header. This allows retrieval of any allowed items with the view results, including rendered rich text items.
 
6) HTTP
 
GET /CurbAppeal.nsf/api/boost/views/76CA31CBCB19C70A8525694F0045C657?metadata=false&additems=Col1,Col2 HTTP/1.1
X-Key-Value: Beachwood, OH
 
6) Java - Unirest
 
Unirest.setTimeouts(0, 0);
HttpResponse<String> response = Unirest.get("
https://geniisupport.com/CurbAppeal.nsf/api/boost/views/76CA31CBCB19C70A8525694F0045C657?metadata=false&additems=Col1,Col2")
  .header("X-Key-Value", "Beachwood, OH")
  .asString();

 
Task 7: Get RTItems (as a list, as individual rtitems in JSON, as rendered HTML pages, or rendered HTML fragments)
Explores some of the different ways to retrieve rich text.
 
7a) HTTP
 
GET /CurbAppeal.nsf/api/boost/documents/524C08BF648BE77A85257B8600798F5B/rtitems HTTP/1.1
 
7a) PowerShell - RestMethod
 
$response = Invoke-RestMethod 'https://geniisupport.com/CurbAppeal.nsf/api/boost/documents/524C08BF648BE77A85257B8600798F5B/rtitems' -Method 'GET' -Headers $headers
$response | ConvertTo-Json
 
7b) HTTP
 
GET /CurbAppeal.nsf/api/boost/documents/524C08BF648BE77A85257B8600798F5B/rtitems/Col1 HTTP/1.1
 
7b) PowerShell - RestMethod
 
$response = Invoke-RestMethod 'https://geniisupport.com/CurbAppeal.nsf/api/boost/documents/524C08BF648BE77A85257B8600798F5B/rtitems/Col1' -Method 'GET' -Headers $headers
$response | ConvertTo-Json
 
7c) HTTP
 
GET /CurbAppeal.nsf/api/boost/documents/524C08BF648BE77A85257B8600798F5B/rtitems/Col1 HTTP/1.1
Accept: text/html
 
7c) PowerShell - RestMethod
 
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Accept", "text/html")

$response = Invoke-RestMethod '
https://geniisupport.com/CurbAppeal.nsf/api/boost/documents/524C08BF648BE77A85257B8600798F5B/rtitems/Col1' -Method 'GET' -Headers $headers
$response | ConvertTo-Json
 
7d) HTTP
 
GET /CurbAppeal.nsf/api/boost/documents/524C08BF648BE77A85257B8600798F5B/rtitems/Col1?fragment=true HTTP/1.1
Accept: text/html
 
7d) PowerShell - RestMethod
 
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Accept", "text/html")

$response = Invoke-RestMethod '
https://geniisupport.com/CurbAppeal.nsf/api/boost/documents/524C08BF648BE77A85257B8600798F5B/rtitems/Col1?fragment=true' -Method 'GET' -Headers $headers
$response | ConvertTo-Json

 
Task 8: Get image from rich text (either as an <img> link or as components in a JSON object, either allowing binary retrieval of the image from the server)
Depending on what exactly is needed, different ways to retrieve an image as a link back to the Domino db.
 
8a) HTTP [Image link as HTML fragment]
 
GET /CurbAppeal.nsf/api/boost/documents/524C08BF648BE77A85257B8600798F5B/rtitems/Photo/chunks/Graphic 1?fragment=true HTTP/1.1
Accept: text/html
 
8a) Python - http.client
 
import http.client

conn = http.client.HTTPSConnection("geniisupport.com")
payload = ''
headers = {
  'Accept': 'text/html'
}
conn.request("GET", "/CurbAppeal.nsf/api/boost/documents/524C08BF648BE77A85257B8600798F5B/rtitems/Photo/chunks/Graphic 1?fragment=true", payload, headers)
res = conn.getresponse()
data = res.read()
print(data.decode("utf-8"))
 
8b) HTTP [Image components as JSON]
 
GET /CurbAppeal.nsf/api/boost/documents/524C08BF648BE77A85257B8600798F5B/rtitems/Photo/chunks/Graphic 1?metadata=false HTTP/1.1
X-Items-List: Photo[$GraphicOffset]=Offset,Photo[$GraphicFormat]=Fmt,Photo[%GraphicHeightPX]=height,Photo[%GraphicWidthPX]=width
 
8b) Python - http.client
 
import http.client

conn = http.client.HTTPSConnection("geniisupport.com")
payload = ''
headers = {
  'X-Items-List': 'Photo[$GraphicOffset]=Offset,Photo[$GraphicFormat]=Fmt,Photo[%GraphicHeightPX]=height,Photo[%GraphicWidthPX]=width'
}
conn.request("GET", "/CurbAppeal.nsf/api/boost/documents/524C08BF648BE77A85257B8600798F5B/rtitems/Photo/chunks/Graphic 1?metadata=false", payload, headers)
res = conn.getresponse()
data = res.read()
print(data.decode("utf-8"))

 
Task 9: Get image data URI
Retrieves an image as an <img> link with the data URI included, so that the entire image is local. The dataimgs parameter shown can be used for any rich text or document retrieval so that the rendered HTML has the images locally available without any requirement for the Domino image to be accessible.
 
9) HTTP
 
GET /CurbAppeal.nsf/api/boost/documents/524C08BF648BE77A85257B8600798F5B/rtitems/Photo/chunks/Graphic 1?fragment=true&dataimgs=max HTTP/1.1
Accept: text/html
 
9) Shell - wget
 
wget --no-check-certificate --quiet \
  --method GET \
  --timeout=0 \
  --header 'Accept: text/html' \
   '
https://geniisupport.com/CurbAppeal.nsf/api/boost/documents/524C08BF648BE77A85257B8600798F5B/rtitems/Photo/chunks/Graphic 1?fragment=true&dataimgs=max'

 
 

Copyright © 2020 Genii Software Ltd.

Wed 23 Dec 2020, 11:23 AM
Inline PNG image
 
In my last post, Out of Notes revisited with Exciton, I described a current customer's requirements for getting data from Domino with Exciton Boost's REST API, and promised I'd make a couple of videos showing how to accomplish these specific tasks using our Curb Appeal sample db. This first video takes you through the whole process using POSTMAN, a free development and testing tool popular with mobile/web developers who need to send requests and record responses. In it, I will show the URL endpoints, URL parameters, and HTTP headers used to accomplish each task.
 
Here are the nine tasks, restated slightly to clarify, and with a brief description of what is being demonstrated. (If you would like, you can request an Exciton Boost evaluation license and follow along the steps yourself. I think could provide a Postman collection to work with.) UpdateCheck out the companion blog post with exact requests and sample code.
 
Task 1: Get list of accessible views
Returns a JSON array of objects, one for each view that is included, implicitly (all views) or explicitly (specified views), in the Exciton configuration db and marked as discoverable. Exciton is strict about data security, so views may be accessible but not discoverable. Views not included are inaccesible and will return a 404 error.
 
Task 2: Get view with columns & metadata
Returns a JSON array of objects, one for each document in the view. Each document includes a link to get the document from the documents collection as well as item values for each view column. Note that there are parameters to page through the view if desired.
 
Task 3: Get filtered view
Returns a JSON array of objects, a subset of the full view filtered by a key match with the first view column. There are alternative ways to filter the view, but this is easiest.
 
Task 4: Get all items from the document
Returns a JSON object containing each item found on the document, or at least all allowed. Items may be explicitly allowed or disallowed in the configuration database, or all may be allowed by default. Some items such as time/date, author and reader fields, and rich text items will be represented by subobjects. Rich text fields are rendered to high fidelity roundtrippable HTML.
 
Task 5: Get only selected items
Using an item list via URL or HTTP header, the request determines which items to returns in the JSON object. In addition to actual items, the request may use Notes formula language to create additional items or may use filtering to get subsets of rich text items.
 
Task 6: Add selected items to view results
Going back to the view, the request can include either additional or replacement items for the view columns using an item list via URL or HTTP header. This allows retrieval of any allowed items with the view results, including rendered rich text items.
 
Task 7: Get RTItems (as a list, as individual rtitems in JSON, as rendered HTML pages, or rendered HTML fragments)
Explores some of the different ways to retrieve rich text.
 
Task 8: Get image from rich text (either as an <img> link or as components in a JSON object, either allowing binary retrieval of the image from the server)
Depending on what exactly is needed, different ways to retrieve an image as a link back to the Domino db.
 
Task 9: Get image data URI
Retrieves an image as an <img> link with the data URI included, so that the entire image is local. The dataimgs parameter shown can be used for any rich text or document retrieval so that the rendered HTML has the images locally available without any requirement for the Domino image to be accessible.
 
Note: Accurate closed captions will be provided shortly.
 
 

Copyright © 2020 Genii Software Ltd.

Tags:

Thu 17 Dec 2020, 11:10 AM
GET request inside POSTMAN
 
Back in 2013, I created a ten part series of blog posts and videos called Out of Notes showing how the Curb Appeal (real estate listings) database with its structured data, computed images, and so forth could be taken from Notes to various formats. Most of those posts are still valid, (though the few with EPUB turned out to be less compelling to many customers). You can still use the Midas LSX to extract data that way, but working with a new Exciton Boost customer made me think about how many challenges they face with a REST API are similar to those I posed with this database. (See bottom of this post for that first video introducing that database and its challenges.)
 
Specific challenges for this Exciton Boost customer include the following (JSON return values unless otherwise indicated):
 
1) (db) GET a list of views configured to be discoverable by the Exciton Configuration database.
2) (view) GET the contents of a specific view from the list using HATEOAS principle. Returned object includes document metadata and column values.
3) (view) GET a filtered set of documents using a primary key. Returned object includes document metadata and column values.
4) (doc) GET all fields from one document.  Returned object includes document metadata and all items, or all allowed items if Exciton Configuration restricts which are accessible. Rich text fields rendered to HTML.
5) (doc) GET subset of items from a document, including mapping of item names and suppression of metadata. Rich text fields rendered to HTML.
6) (view) GET a filtered set of documents using a primary key. Returned object includes subset of items from a document, including mapping of item names and suppression of metadata. Rich text fields rendered to HTML.
7) (rtfield) GET single rich text fields rendered to HTML.
8) (image) GET single image from rich text field as binary
9) (image) GET single image from rich text field as data URI
 
There are lots more things we could do, but these show a number of features of the Exciton Boost REST API. I'll put together a video showing these steps using POSTMAN, then a second video showing how we can take advantage of these in JavaScript. But first, take a quick look at the database from that original 2013 challenge video, and imagine the databases in your domain which include various features like these.
 
Note: Accurate closed captions have been provided, and if English is not your first language, you might want to turn on the auto-translate.
 
  

Copyright © 2020 Genii Software Ltd.

Tags:

Mon 7 Dec 2020, 04:47 PM
Inline JPEG image
 
 
On Friday, Heiko Voight wrote a post called domino-db, proton and Date/Time values. It makes interesting reading, but the gist of it is that domino-db can be mighty persnickity about dates, and not very respectful of time zones. As an example, Heiko pointed out the two dates below. The invalid one was created using a standard JavaScript function that happens to return milliseconds (3 decimal places) rather than hundredths (two decimal places). Rather than either ignore the extra precision or round it off (worthy of debate which is better), Proton simply chokes on it.
 
'2013-03-01T14:09:01.009Z' // Invalid (too precise)
'2013-03-01T14:09:01.01Z' // Valid
 
 
As it typical with domino-db and Proton, this doesn't just return that it is invalid, it pukes all over the screen. (See above.) Even worse in its own way, the time zone information included in the date was ignored, thus interpreting every date as if it were GMT (nice for Gab and Tim, but not many of the rest of us).
 
Out of curiosity, I checked to see what Exciton Boost 4.5.3 did with these dates. We use a date conversion function that is a bit more flexible. It works properly with any of the following:
 
2013-03-01T14:09:01.009Z
2013-03-01T14:09:01.01Z
2013-03-01T14:09:01.0Z
2013-03-01T14:09:01Z
2013-03T14:09:01.009Z
20130301T14:09:01.009Z
2013-03-01T140901.009Z
20130301T140901.009Z
2013-03-01T14:09:01.009-0400
2013-03-01T14:09:01.009-04
2013-03-01T14:09:01.009-0430
2013-03-01
20130301
 
and quite a few other variations, as it happens, including variants such as:
 
Mon,  7 Dec 2020 15:06:31 -0500
Today
Now
@Adjust(@Today; -1; 0; 0; 0; 0; 0);
 
The key concept is that we should be liberal in what we receive and conservative is what we send. Plus, we shouldn't puke all over. If we do manage to give a date format that is not accepted, such as
 
Mom's birthday
 
we are really okay with a simple invalid request mentioning that the date could not be interpreted. (No, mom, we aren't adding that in 4.5.4.)

Copyright © 2020 Genii Software Ltd.

Tags:

Thu 3 Dec 2020, 11:49 AM
 
Inline PNG image
 
Two related topics which come up a lot with customers interested in our Exciton Boost  product are authentication and authorization (aka access). The two terms are often used somewhat interchangeably, but they are quite distinct.
 
Here's an overview of some differences between authentication and authorization. This table is adapted from an excellent discussion of the topic on Auth0's website at Authentication and Authorization.
 
Authentication (Domino and/or IAM service)
Authorization (Domino and Exciton Boost)
Determines whether users are who they claim to be
Determines what users can and cannot access
Uses passwords, security questions, 2FA, etc. to validate the user's credentials
Verifies access using access control lists and other policies and rules
Usually done before authorization, or sometimes after a very thin layer authorization (if nobody has access to a server, there's no need to authenticate before determining that).
Usually done after successful authentication. If anonymous access is allowed, authorization is based on what is considered accessible to anyone, often very limited.
Domino session authentication usually relies on a session token in a cookie while IAM uses an ID token plus secret.
Either a session token is passed via HTTP (as cookie) or info is transmitted through an Access Token.
Domino session authentication uses its own NAB or LDAP or whatever, while IAM is governed by an OpenID Connect (OIDC) protocol. (I think. I am not great with these admin details.)
Authorization relies on Domino security first, via a cascading set of factors such as maximum internet access and ACLs. Access using Exciton adds an additional layer of more finely tuned access based on the Exciton Configuration database.
 
Basically, the first step is to prove who you are to Domino. Then, once Domino is convinced you are you, both Domino and Exciton have to decide whether you have sufficient access (authorization) to do what you want to do. Finally, you do it. (All this sounds complicated, but it happens in a blink of the eye.)
 
Domino's ACL is well known. Exciton's access is defined by a configuration database which defines which databases, forms, and views are accessible, and whether they can be used with the REST API or the RPC API. Specific fields on those forms can be defined as accessible or inaccessible (include or exclude field lists). Thus, while Domino Access Services will let you set almost any fields with almost any form, or no form, Exciton Boost makes sure you are only able to create, read, update, and delete fields and documents that you are supposed to.

Copyright © 2020 Genii Software Ltd.

Tags: