Ben Langhinrichs

Photograph of Ben Langhinrichs

E-mail address - Ben Langhinrichs






October, 2020
SMTWTFS
    01 02 03
04 05 06 07 08 09 10
11 12 13 14 15 16 17
18 19 20 21 22 23 24
25 26 27 28 29 30 31

Search the weblog





























Genii Weblog


Civility in critiquing the ideas of others is no vice. Rudeness in defending your own ideas is no virtue.


Mon 26 Oct 2020, 05:05 PM
Inline JPEG image
 
At Genii Software, we help a lot of companies get data from Domino applications, whether for migration (using the Midas LSX) or for use in mobile/web apps (using Exciton Boost, one of our Midas products, or a bespoke addin created for the customer). While we are best known for our high fidelity rich text rendering, a lot of application data is not rich text. But that doesn't mean it isn't complex. In this series, I will show ways that the complex is handled simply by both the Midas LSX and Exciton Boost.
 
A couple of weeks ago, we had a migration customer ask if we could change the way their data was extracted. In this case, we were going to CSV. I can't show their data or structure, but it contained a feature I lot of you will recognize: many, many fields correlated with each other by table rows. Since the customer wanted to retain the row-like association of the various items, we used Repeat By Item, a feature which allows you to export the same document multiple times with each element of a multi-value field or formula, matching up parallel items. We can include the UNID on every line or not, depending on whether we want to maintain the association the data had in Domino.
 
To show how this often looks in a Domino application, I created a simple form and view to display the box scores used to record player stats in baseball. (You could do the same with cricket or another sport, of course.) Here's a game played recently with a small set of box scores. This is from the Cleveland Indian's point-of-view, and there is a second document for the same game from the Chicago White Sox point-of-view. The goal might be to create a spreadsheet such as that above or to export the data into SharePoint or almost anything else. CSV imports are fairly ubiquitous. 
 
In our bare bones Domino document, we have the general fields followed by a table of the stats for each player as they entered the rotation. 
 
Domino document
 
In our bare bones Domino document, we have the general fields followed by a table of the stats for each player as they entered the rotation. 
 
Inline JPEG image
 
Domino form
 
In our form, we have a sight all too familiar to long-term Notes/Domino developers. Each row has individual fields, sometimes more than one to a table cell. The names are related by some naming convention. In our customer's form, there were over 700 named fields.
 
Inline JPEG image
 
 
Using the Midas LSX with Midas Exports, we accomplish this by filling out a simple form. No scripting is required, though formulas are allowed and are used in this case. The key parts of the form are here:
 
Inline JPEG image
and below that, the split by item (though this version of the db has a spelling error. I mean "spliting"? Yikes.):
 
Inline JPEG image
 
If we run the agent in our database, we get this CSV
 
Inline JPEG image
 
Opened in Excel and tweaked slightly so we can see it better, we get the desired spreadsheet shown at the top of this post. 
 
This worked well for the customer, generating several million rows of data to import into their new system. We also generated high fidelity renderings of the forms, but that was for a different purpose.
 
Request a Midas LSX evaluation license and see if it meets your migration needs.
 
As it happens, we can do this exact same rendering using Exciton Boost's REST API.  Instead of an agent, we used a GET request to display one of the views: 
 
 
In the request, we set the Accept header to text/csv. We set the X-Items-List to the same formula as in the form for the Midas export:
 
Team+@If(Home="";" at ";" (h) vs. ")+OppTeam+" on "+@Text(GameDate)=Game,(Num:Num_1:Num_2:Num_3:Num_4:Num_5:Num_6:Num_7:Num_8:Num_9:Num_10:Num_11:Num_12:Num_13)=Num,(Player:Player_1:Player_2:Player_3:Player_4:Player_5:Player_6:Player_7:Player_8:Player_9:Player_10:Player_11:Player_12:Player_13)=Name,(Pos:Pos_1:Pos_2:Pos_3:Pos_4:Pos_5:Pos_6:Pos_7:Pos_8:Pos_9:Pos_10:Pos_11:Pos_12:Pos_13)=Position,(AB:AB_1:AB_2:AB_3:AB_4:AB_5:AB_6:AB_7:AB_8:AB_9:AB_10:AB_11:AB_12:AB_13)=AB,(Rs:Rs_1:Rs_2:Rs_3:Rs_4:Rs_5:Rs_6:Rs_7:Rs_8:Rs_9:Rs_10:Rs_11:Rs_12:Rs_13)=R,(Hs:Hs_1:Hs_2:Hs_3:Hs_4:Hs_5:Hs_6:Hs_7:Hs_8:Hs_9:Hs_10:Hs_11:Hs_12:Hs_13)=H,(BB:BB_1:BB_2:BB_3:BB_4:BB_5:BB_6:BB_7:BB_8:BB_9:BB_10:BB_11:BB_12:BB_13)=BB,(RBI:RBI_1:RBI_2:RBI_3:RBI_4:RBI_5:RBI_6:RBI_7:RBI_8:RBI_9:RBI_10:RBI_11:RBI_12:RBI_13)=RBI
 
The response is:
 
201 OK
Game,Num,Name,Position,AB,R,H,BB,RBI
Chicago White Sox at Cleveland Indians on 09/24/2020,7,Tim Anderson,SS,4,0,0,0,0
,10,Yoan Moncada,3B,4,1,1,0,0
,24,Ysmani Grandal,C,4,0,0,0,1
,79,José Abreu,1B,3,1,0,1,0
,74,Ely Jiménez,LF,3,1,1,0,0
,15,Adam Engel,LF,0,0,0,0,0
,33,James McCann,PH,1,0,0,0,0
,23,Edwin Encarnación,DH,3,0,0,0,0
,30,Nomar Mazara,RF,3,0,1,0,2
,5,Yolmer Sánchez,2B,2,1,1,1,1
,32,Jarrod Dyson,CF,3,0,0,0,0
Cleveland Indians (h) vs. Chicago White Sox on 09/24/2020,12,Francisco Lindor,SS,3,0,0,1,0
,7,César Hernández,3B,4,1,3,0,3
,11,Jose Ramirez,3B,3,0,2,1,2
,41,Carlos Santana,1B,4,0,0,0,0
,32,Franmil Reyes,DH,4,0,1,0,0
,8,Jordan Luplow,RF,2,0,0,0,0
,31,Josh Naylor,PH-LF,2,0,1,0,0
,35,Oscar Mercado,LF,2,0,0,0,0
,30,Tyler Naquin,PH-RF,2,1,1,0,0
,55,Roberto Pérez,C,2,1,1,0,0
,6,Mike Freeman,PH,1,1,1,0,0
,17,Austin Hedges,C,1,0,0,0,0
,0,Delino DeShields,CF,3,1,0,0,0
 
Or we could switch to JSON or XML and use those instead, simply by changing the Accept header. It's that simple.
 
Request an Exciton Boost evaluation license and try it out in your modern web/mobile apps.
 

Copyright 2020 Genii Software Ltd.

Tags:

Wed 7 Oct 2020, 05:40 PM
 
Inline PNG image
 
I will be presenting at CollabSphere 2020 Live (online conference) at the end of the month. No schedule yet, but I'll let you know when I know. My session details are below:
 
Track: DevOps

Custom Domino Extensions in a Modern AppDev World
 
Building extensions to Domino with the C API may seem like an
old-fashioned idea, but building specialized REST APIs, Node.js extensions,
and more is surprisingly easy and powerful. Whether you want to expose Node
libraries to Domino or encapsulate complex business logic in a simple API
for use in mobile apps, or add a special security layer to your existing
applicationss, there's always room to expand Domino around the edges if you
know where to start.
 
I hope to see you (virtually) there. Details on the conference, including how to register for this free, but so valuable, conference can be found at CollabSphere 2020 Home.

Copyright 2020 Genii Software Ltd.

Tags:

Tue 22 Sep 2020, 11:52 AM
 
 
Inline JPEG image
 
This series of posts is meant to introduce Exciton Boost and provide some comparisons with Domino Access Services. Today, I'm going to talk about the biggest barrier customers report with Domino Access Services: security and integrity.
 
Domino Access Services has a very open (some would say insecure) approach to access. A database must be enabled, which is good. Domino security is respected, at least mostly. There are some security holes with anonymous use, but if you are allowing anonymous access to your database, you probably aren't too worried about security.
 
But there are some serious access control issues which prevent companies from using Domino Access Services. Among them:
 
1) Wide open discovery of databases. Using the endpoint http://example.com/api/data gives you a complete list of every database on the server, including its title, filename, and what template it is based on. Even if you have hundreds of dbs and only one is accessible, they are all included. While obscurity is not security, there is no reason to invite outsiders to view all of this. What if you have client databases by directory, a fairly common approach. Now, for no good reason, your client list is available to a casual browser.
 
Exciton Boost only displays databases that are a) available for Exciton Boost, and b) marked as discoverable in the Exciton configuration database.
 
 
2) Open discovery of views inside a database. Using the endpoint http://example.com/MyDb.nsf/api/data/collections gives you a complete list of every view and folder, private or public, in the database. Even if the database is only open to Domino Access Services to post anonymous public documents, every view and folder is revealed.
 
Exciton Boost only displays views that are explicitly available for Exciton Boost. Everything else is private, as it should be.
 
 
3) No control over forms created. With Domino Access Services, you can create documents with no form name or with any form name, including invalid names or hidden forms that are not meant to be created by users.
 
Exciton Boost only allows forms to be created that are explicitly available for Exciton Boost. You maintain complete control.
 
 
4) Little control over items created. With Domino Access Services, you can create almost any item you like so long as it does not start with a dollar sign. Invalid item names, items meant to be different types, etc. are all open for creating and updating. This invites both mischief and mistakes.
 
Exciton Boost allows as much or little control over the fields to be created based on the form. You can leave it wide open, partly open, or locked down.
 
 
5) Form validation is left in client control. With Domino Access Services, the caller can choose to compute-with-form or not. If it is not chosen, none of the field validation or translation formulas are run, and computed fields are not computed. This may leave the document in an unstable or non-viable state.
 
Exciton Boost allows the form to be marked as Always compute-with-form if desired. You can be sure the fields are in their correct state without the client app being able to disable the setting.
 
 
These are just some of the access control measures we have implemented in Exciton Boost. There are also Query Save and Query Open agents as I wrote about in Exciton Boost - Secret Agents protecting your data
Access levels are also set at the database, form, and view level that control creating, updating, and deleting documents. These never override Domino security, they simply augment it so that you have control over what mobile apps can do. The database form is shown below as an example. This is from the Exciton Configuration database which control access through both Exciton Boost and Exciton Power.
 
Inline JPEG image
 
While one would hope that there aren't really barbarians at the gate, Exciton Boost ensures that you have control over what the barbarians can do once you invite them in.
.
 
==> Request a free Exciton Boost evaluation license, and we'll send you the license and a link to the software as soon as it is released. <==
 
 

Copyright 2020 Genii Software Ltd.

Tags:

Thu 17 Sep 2020, 02:45 PM
 
Inline JPEG image
 
 
I'm spending the week talking about Exciton Boost's features and some comparisons with Domino Access Services. Today, I'll talk about rich text rendering to HTML. It doesn't sound sexy, and you may think it is just about looks, but data is at stake here, so pay attention.
 
  • Key Concept: Content delivered should be the same content requested,
  • Differentiator: Exciton Boost uses the Midas engine for rendering, providing the most accurate and highest fidelity available anywhere.
There are multiple rich text to HTML rendering engines in Notes/Domino, and they all have problems. By using the award winning Midas engine foir our rendering, Exciton Boost provides high fidelity HTML that matches the content and look-and-feel of the rich text in Notes. I will show two examples, both fairly common scenarios that exhibit data loss or mangling, and both of which could cause confusion on the client app. These use the Domino Access Services REST API available in Domino 11.0.1.
 
Example 1) Nested List Confusion
 
Nesting in lists in Notes is created by indentation, and people are often sloppy about how much they indent.
 
Question to ask yourself when looking at the results of each. Is point D "Legal" or "Benefits"?
 
 
Nested List Confusion - What it looks like in Notes 11.0.1 client
 
Inline JPEG image
 
 
Nested List Confusion - What it looks like after GET from Domino Access Services 11.0.1
 
Inline JPEG image
 
 
Nested List Confusion - What it looks like after GET from Exciton Boost 4.5.0
 
Inline JPEG image
 
 
Example 2) Form Elements
 
Whether through forwarding or RenderToRTItem or define a field as rendering the parent document, it has always been quite common to see "form elements" such as radio buttons and combo boxes in rich text fields. Many applications even use this as part of the workflow. So, while the look of the tables is notably different, try to ignore that and focus more on the data
 
Questions to ask yourself when looking at the results of each: What is the value next to "Product"? Is this "Chargeable?" Is a Reminder Needed?" What is the "Status"? These vital pieces of information should be discernible from the HTML generated, no matter what the appearance. 
 
 
Form Elements - What it looks like in Notes 11.0.1 client
 
Inline JPEG image
 
 
Form Elements - What it looks like after GET from Domino Access Services 11.0.1
 
Inline JPEG image
 
 
Form Elements - What it looks like after GET from Exciton Boost 4.5.0
 
Inline JPEG image
 
 
 
There are many more examples of data loss, corruption, obscuring, and that is even before considering the professionalism of the rendered rich text.
.
 
==> Request a free Exciton Boost evaluation license, and we'll send you the license and a link to the software as soon as it is released. <==
 
 

Copyright 2020 Genii Software Ltd.

Tags:

Tue 15 Sep 2020, 03:27 PM
Inline PNG image
I was curious how a more complex formula would work with Exciton Boost. While this may look dense, it is a real one I created for the Lotusphere Sessions db back in 2004. Here it is in formula language without the escaped quotes and with colors to guide the eye. (Also, you can copy this, as it is text rather than an image.)
 
sorted := @Sort(Speaker; [CustomSort];
@If(
   @Do(R:=@Transform(@Explode($A; " "); "W";
      @If(@Matches(W; "{a-z}*"); W+@RightBack($A; W)+", "+@LeftBack($A; W); @Nothing));
         @If(@Trim(R) != ""@Trim(R)[1]; @RightBack($A; " ")+", "+@LeftBack($A; " "))) >
   @Do(R:=@Transform(@Explode($B; " "); "W";
      @If(@Matches(W; "{a-z}*"); W+@RightBack($B; W)+", "+@LeftBack($B; W); @Nothing));
         @If(@Trim(R) != ""@Trim(R)[1]; @RightBack($B; " ")+", "+@LeftBack($B; " ")));
   @True@False));
 
If you look in the properties after POSTing this, you see that the Speaker item has a text list as specified.
 
Inline JPEG image
 
The SpeakersSorted item has the same text list, now sorted by last name. A tweak would make this sort with German sorting rules (van der Welten after Veems).
 
Inline JPEG image
 
 
 
 
==> Request a free Exciton Boost evaluation license, and we'll send you the license and a link to the software as soon as it is released. <==
 
 

Copyright 2020 Genii Software Ltd.

Tags:

Tue 15 Sep 2020, 11:18 AM
 
Inline JPEG image
 
 
With Exciton Boost out this week, I'm spending the week talking about its features and some comparisons with Domino Access Services. Today, I'll talk about formulas, specifically Notes formula language, and how access to formula evaluation can empower your mobile and web apps in both the REST API and JSON-RPC API.
 
  • Key Concept One: Not all information is available to the client, nor should it be,
  • Differentiator One: Exciton Boost allows client apps to express values as Notes formulas, and allow the Domino server to evaluate and use the data which the client doesn't have.
 
  • Key Concept Two: Some operations require intermediate results, but should not trigger repeated saves or repeated calls back and forth from the client and server,
  • Differentiator Two: Exciton Boost allows intermediate results to be stored and then retrieved using Notes formulas
Because the API in Domino Access Services treats Domino as a simple data store, calculations usually require data to be brought back to the client. If this means intermediate results, it requires multiple requests which is both inefficient and leaves the document on the server in a partially updated state for too long.
 
 
Setting field values in REST API using formulas
 
In Domino Access Services, using POST to create and PUT/PATCH to update, field values must be known to the client and expressed explicitly. Alternatively, the form must contain all the fixed formulas and must be evaluated to get the results, which is less flexible.
 
In Exciton Boost, field values may be set directly or evaluated by a server-side formula. Let's look at an example where we want the request to reflect the abbreviated, not common, name of the logged in user, and we don't want to have to modify the request payload. In the first line below, we only know the name used to log in, and we have to alter the payload to ensure it is used. In the second line, we let the server evaluate the formula to determine the abbreviated name, which it does know.
 
Explicit => "Requestor": "Ben Langhinrichs"
 
Formula => "Requestor": {"@evaluate": "@Name([Abbrev]; @UserName)"}
 
Another example might be where an item number is known, but we want to use the item name. This requires a lookup to another database. 
 
{
"ItemNo": "REDM01",
"ItemDescr": {"@evaluate": "@DbLookup(\"\"; \"\":\"Orders.nsf\"; \"(ItemLU)\"; ItemNo; 2)"}
}
 
In this example, the ItemNo is saved as a value on the document, and is also used as an intermediate value in the formula for ItemDescr. If you retrieved the document afterwards, the values would be returned as
 
{
"ItemNo": "REDM01",
"ItemDescr": "Redmond Real Salt - Ancient Fine Sea Salt, Unrefined"
}
 
For those worried about opening up the power of @DbLookup and so forth, we are putting various controls in place to make sure they are not used to access data which is off limits, above and beyond the control based on Domino security.
 
Setting parameter values in JSON-RPC API using formulas
 
In Exciton Boost's JSON-RPC API, methods are called with specified parameters. Let's use the example from yesterday with agents. 
 
Request (using POST): 
{"jsonrpc": "2.0", "method": "doc.runAgent", "params": {"@evaluate": "@If(SalesTotal >= SalesTarget; \"Reward profitability\"; \"Punish failure\")"}, "id": "msg"}
 
Response (with 200 OK):
{"jsonrpc": "2.0", "result": "Loser! You missed your sales target. No soup for you!", "id": "msg"}
 
Different agents were run depending on whether the SalesTotal was greater than the SalesTarget, though the client never needs to know what either value is. The return value in the response is the output, if any, from Print statements in the agent. I use it here to send a message.
 
 
Another example is where we have a batch of RPC calls. By setting a url paramater, we can specify that return values with string ids be saved as intermediate field values. We are running this on the Body field of a specific document.
 
{"jsonrpc":"2.0", "method":"rtc.getCount", "params": "Graphic", "id": "ImageCount"}, 
{"jsonrpc":"2.0", "method":"rtc.getCount", "params": "Doclink", "id": "DoclinkCount"}, 
{"jsonrpc":"2.0", "method":"rtc.appendTable", "params": [1, 2, "", {"@evaluate":"\"Doclinks: \"+DoclinkCount"}, {"@evaluate":"\"Images: \"+ImageCount"}]}
 
 
The first two calls use Exciton Boost methods to get the number of images and doclinks in the Body field, and saved in numeric fields called ImageCount and DoclinkCount. Then, we append a table at the end of the Body field with one row and two columns. The values of the columns show the number of doclinks and images, using those intermediate fields from the document. 
 
These are fairly simple examples, but they show how powerful formula evaluation can be, and how intermediate results allow a series of actions based on earlier actions.
 
==> Request a free Exciton Boost evaluation license, and we'll send you the license and a link to the software as soon as it is released. <==
 
 

Copyright 2020 Genii Software Ltd.

Tags: