Genii Weblog

Civility in critiquing the ideas of others is no vice. Rudeness in defending your own ideas is no virtue.

Mon 22 Sep 2003, 01:58 AM
Rich Text 101 logo
While many other things can be found in a rich text field besides text, such as tablesdoclinksimages and more, a great deal of rich text is just that, text.  In my previous article on paragraphs, I mentioned that even in the properties box listed as "Text", most of the properties are actually paragraph properties.  In fact, only the very first tab is actually related to text properties.
Test properties nox

There are some important differences between paragraph attributes and text attributes.  Text attributes are available for any run of text, so that the string "The brown fox jumped over the haystack" is actually three separate runs of text with different text attributes. Paragraph attributes must be applied to an entire paragraph, so you can never have two sets of different paragraph attributes on the same line.  This is why when two fields must appear on the same line but should have different hide-when formulas (paragraph attribute), the solution is to put each in its own table cell.  Every table cell starts with a new paragraph.

Font Faces
Any text shown on a form must appear in some font face.  For text and number and date and keyword fields, this font face is determined on the  form, which is also rich text as described in my paragraph article.  For rich text, this font face is determined by default by the form, but may change at will within the rich text.  A sentence, a phrase, a word or even  single letter can be in a different font face, such as Comic Sans MS.  As I will mention below, the fact that a font face is defined for a rich text run does not always mean it can be displayed by the person reading the rich text.  For this reason, there are four default font faces, Default Sans Serif, Default SerifDefault MultiLingual and Default Monospace.  (For those who used Notes in earlier versions, these used to be called Helv, Roman, Typewriter and Courier).  Text set to be displayed in one of these four fonts will actually be displayed in the font that is defined as default for that type.  The default fonts can be set through File - Preferences - User Preferences under the Default Fonts button.

$Fonts and the Case of the Messed Up Font Faces
Long time users of Lotus Notes can usually recall many cases of font faces that changed mysteriously.  Often, all fonts other than the default four were all converted to monospace with little explanation.  Sometimes, one rich text field was appended to another with catastrophic results to the font faces of either the new or old rich text.  While some of these problems have been resolved in R5 and ND6, it is still possible to see this happen when a rich text item is copied as an item, for example.

The reason for all of these problems is the $Fonts field.  In the hazy mist of time, it was decided that font faces should not be referenced directly in rich text.  Rather, each of the default four font faces could be referred to by a number between zero and four (there is one left over that is not used), and each new font that was used would be added to the $Fonts item, which contains a font table.  Thus, the first new font would cause a $Fonts item to be added, and the face number would be five.  The second would be appended to the $Fonts item and the face number would be a six.  This way, even if the two new fonts were used liberally throughout one or more rich text fields, each reference would only contain the face number.

While this was efficient, it caused immediate problems.  Imagine that a form (Form A) is created using Comic Sans MS as its first new font.  (Remember that forms are rich text as well.) The face number would then be five.  Now, imagine that a second form (Form B) has no special font faces, so it has no $Fonts item.  Finally, imagine that a document is created using the second form and a special font, let's say Verdana, is used in the rich text.  Since that is the first special font, a $Fonts item is created on the document, with a face number of five.  What happens when the document with the $Fonts item is displayed using not Form B, but rather Form A.  On Form A, the face number five means Comics Sans MS, but in the $Fonts item that was carried with the document, the face number five means Verdana.

What should happen in these cases is that the font tables get merged and all font face numbers get updated (which is what the Midas Rich Text LSX does), but what often happened in previous release was that one $Fonts item overwrote the other, and the font face was misidentified.  In the similar current case where the rich text is copied as an item without the $Fonts being copied as well, the font face numbers aren't recognized at all, and are often treated as monospace.  In yet another variation, a well meaning developer copies the $Fonts item with the rich text, neglecting the fact that there might already be a $Fonts item, and thus all the original font associations are broken.  Yikes!

Fortunately, you can avoid most of these problems in ND6 by using the AppendRTItem, which seems to do a fairly good job of merging the font tables and updating the fonts.  Of course, if you still have trouble, you can fall back on using Midas in almost any version and get the same merging and fixup.

Text Attributes
While the font face and point size may be loosely described as "text attributes", what is meant in this case are the attributes such as BoldItalicsUnderlineSuperScriptSubScriptStrikethroughShadow, Emboss and Extrude.  These text attributes can be combined (even to the point of silliness), except for the Shadow, Emboss and Extrude, which are mutually exclusive.  It is worth noting that these last three attributes are also only displayed in the Notes client.  There is no matching text attribute in standard HTML, although there are some close equivalences in CSS.  In general, it is not wise to use text attributes too extensively, as they tend to be wearing on the eyes.  On the other hand, these attributes can be printed, while text colors are often lost on the printed page.

Text Colors
In earlier releases of Notes, there were only sixteen colors available for text.  These colors have constant names which are used with the NotesRichTextStyle class in Notes.  Additional colors were available from a color pallette.  In R5, RGB colors were added as well, so now text can be almost any color available through RGB colors.  It should be cautioned that these colors are often lost in the translation to MIME when mail is sent through the internet, so when the rich text is intended for that use, it may make more sense to add emphasis through text attributes.

A great advantage of the use of RGB colors is that there is no need to match the pallette used in Notes with that used on the web, as was necessary in R4 to get color fidelity.

Point Size
Not much to be said here, except that it should be noted that not all fonts allow all point sizes.  This is especially worth noting for the default fonts.  If my Default Sans Serif font is Arial, I may set the point size to 4pt to make a very small line of text, but if the reader's Default Sans Serif doesn't display anything smaller than 8pt, the text won't look small the way I intended.  It is a good rule of thumb not to use anything but standard font sizes, usually 8, 10, 12, 14, 18, with the default four font faces.  (Obviously, if you can be quite sure of the font used in your company, you can disregard this rule)

Pass-Thru HTML
An additional attribute of rich text is the Pass-Thru HTML attribute, available only through the Text menu.  This can be used to mark raw HTML code that should be passed to the browser, but it should be avoided for mixed Notes client/web browser applications, as the HTML appears as is to the Notes client.  It should also be noted that this menu item is greyed out if there is not a Designer client on the workstation.

Be careful when using the Pass-Thru HTML, as it tends to mask returns.  If you just want one line to have the attribute, and want a return to follow, make sure the last character of the line does not have the attribute.  On the other hand, if you want to avoid a return after the marked line, make sure the attribute extends to the very end of the line, and possible to the first character of the next.

This is an underutilized feature added in R5 that allows you to highlight text the way a textbook is highlighted, by turning on the Text - Highlighter - Use Pink Highlighter (or Yellow or Blue) and drawing the mouse over the desired text.  One advantage to this form of highlighting is that it does not effect any of the regular text attributes.  One disadvantage is that it is hard to control and not obvious how to clear the highlight attribute once it is in place.

Just so you know, the way to clear the highlighter is to leave the Highlighter menu item checked and then hold down the Shift key.  Going backward over the highlighted text will erase the highlight. [Note: It has been pointed out by Jan Van Puyvelde that the Shift key does not need to be held down.  Just going backward will erase the highlight.  I am not sure whether this was always the case, but it highlights the fact that I don't even know for sure how to undo this properly]

A disadvantage of the highlighter is that Domino will not translate it to HTML, since there is not native HTML method for highlighting.  There is a CSS property that could display this, but it is not used by Domino in R5 or ND6.  For the moment, this feature should not be used for web applications, mixed applications or where the rich text is to be mailed outside of the Notes environment.

Permanent Pen
This is a great feature added in R5 that seems to be widely used in some organizations and completely unknown in others.  The way it works is that you can turn on a combination of text attributes, text color, point size and font face, and then use the menu item Permanent Pen - Set Permanent Pen Style.  From that point on, and this is a workstation dependent feature, when you click on the smart icon that looks like a pen point, or use the Permanent Pen - Use Permanent Pen Style menu item, your typing will follow the saved style.  The best use for this is when editing a document, since you can easily identify your changes and edits.  Among groups that use this frequently, it is common for each member to pick his or her own color and style, after which just a glance at a revised document can tell any member who has edited and what changes were made.

Language tagging
This text feature was added in ND6, and allows you to tag text as belong to a specific language.  Thus, a phrase which was appropriate in Spanish but which did not translate well to French could be replaced by a different phrase which made sense in French.  This is a powerful feature, but be careful, as it is hard to read the multiple phrases and make sure all get updated simultaneously.

Bi-directional text
Some languages, such as Hebrew, are read from right to left rather than left to right.  This is only slightly visible, as the text is usually aligned right rather than left, but the font face is the only sure way to tell, as there is no text property exposed for bi-directional text.  For example, Time New Roman (Hebrew) is right to left, but that is only obvious if you know it ahead of time.

Rich text can appear as little more than plain text, but can also be immensely rich and complex.  While overuse of these features is not likely to be pleasant, it is also worth remembering that features such as Permanent Pen and Highlighting, as well as Language Tagging, are all ways to use rich text as part of your application and part of your feature set.  Too little use of font faces, colors and text attributes can leave your text harder to read and harder to decipher.  Judicious use of attributes can guide the reader to important points.  Rich text is a resource to be used wisely, not under or overutilized.

Related Articles
A number of the articles in the Rich Text 101 series contain information closely related to this topic.  Take a look at any of these articles to learn more about the text in rich text:

Copyright 2003 Genii Software Ltd.


Fri 19 Sep 2003, 02:21 PM
On Signal vs. Noise today, there is an excellent post on the dilemma of designing an application that will be full of data, but delivering it empty.  Take a blog template for example.  When you see a blog in use, it has lots of links and articles and calendar entries and looks great.  Then, you start your own blog and face devestating emptiness.  There are no months where the months go until you have created some posts.  The views come up with No Documents Found messages.  There are no links.

Unfortunately, we as developers are uually working with a full set of data, possibly faked, but still full.  The customer or potential customer starts with a blank slate at the exact moment when they need to evaluate and make a purchase/pay decision.  What is the best way to address this?  We ignore it at our peril.  Aye!

Copyright 2003 Genii Software Ltd.

Fri 19 Sep 2003, 12:35 AM
OK, I've probably bored everybody to death on the subject of @Midas Formulas, and I don't know whether anyone is reading these or not anymore (Actually, that isn't quite true.  Due to the wonders of Nedstat, I know that yesterday 52 people read my blog, and they read 103 pages, but I don't know whether they were happy about it).  Anyway, here's a promise.  Stick with me through just one more, and I'll try to write a nice juicy Rich Text 101 article tomorrow.  Is it a deal?

I feel that it is very important when writing an extension to a language to be consistent with the way things are done in that language (except when they are done wrong, and then all bets are off).  When writing the Midas Rich Text LSX, I wanted to make it easy, but I also wanted to make it work the way LotusScript seemed to work.  [Note: I think the Notes 6 developers missed this when they added the new rich text classes, and even more the XML classes.  Instead, they tried to make the LotusScript classes consistent with the C++ API classes, which I feel was a big mistake.  Anyway, that is the topic for another blog one day.]

Similarly, when working with the formula language, I try to stay consistent with the way the current @formulas are written.  But what am I to do when the formula language itself isn't consistent.  Take, for example, @Environment.  This function gets or sets the value of an environment variable.  That's right, it does either.

Using @Environment to get the value of an environment variable

If your formula says:


and there is an environment variable called LastReturnState with the value Ohio, then the return value is Ohio.

Using @Environment to set the value of an environment variable

If your formula says:

@Environment("LastReturnState"; "Michigan")

then this sets the value of the LastReturnState environment variable to the value Michigan.

So why isn't there a single function @DocValue?

To be consistent with @Environment, which has been around for a gazillion years, there should be a single function called @DocField.  It would have two required parameters, and one options.  @DocField(unid; fieldname) would return the value, the way @GetDocField does now.  @DocField(unid; fieldname; newValue) would set the value, the way @SetDocField does now.  Similar get/set pairs are found with @Field, @ProfileField, @TargetFrame.  In fact, they even added a new function called @SetEnvironment which does the same thing as the latter use of @Environment.  One can only presume that a new developer was working on formulas by this time, and that new developer was more comfortable with Java than with C++.  Or, it is simply possible that the developers agreed with Ralph Waldo Emerson about foolish consistency.

OK, Ben, what's your point?

I'm getting there, I'm getting there.  Let's return to the intriguing (to me) concept of treating rich text fields as data, especially by using @DbColumn/@DbLookup on rich text tables.  I have blogged about how to do this with @Midas.  I have blogged about how to use the same technique in Java.  Now, let's take it one step further.  

I could define a function using @DbCommand to set a column of data in a rich text table, but it doesn't seem consistent.  What occurs to me is, why not be consistent with @Enviornment?  Why not use the syntax

@DbColumn("Midas" : "NoCache" ; serverdatabase
                              @DocumentUniqueID ; columnNumber )

to get the value of a column, and use

@DbColumn("Midas" : "NoCache" ; serverdatabase
                             @DocumentUniqueID ; columnNumber ; columnValues )

to set the value of a column.  Similarly, use:

@DbLookup("Midas" : "NoCache" ; serverdatabase
                             @DocumentUniqueIDfieldName ) or
@DbLookup("Midas" : "NoCache" ; serverdatabase
                             @DocumentUniqueIDkeycolumnNumber )
to get the value of a particular column or field based on a key, then use

@DbLookup("Midas" : "NoCache" ; serverdatabase
                             @DocumentUniqueID viewkeyfieldNamefieldValue ) or
@DbLookup("Midas" : "NoCache" ; serverdatabase
                            @DocumentUniqueIDkeycolumnNumber columnValues )

In our table from  before, we took a table such as this one from the Partnerworld site and put it in a rich text field:

IBM Value Package for Developers
Annual fee
North America, Europe, Middle East, Africa 
Latin America 
Asia Pacific region

Then, we specified a command such as:

prices := @DbColumn("Midas"; @DbName; @DocumentUniqueID; "Body";
                                             2; "Format='Currency' SkipTitle='Yes' ");

and the prices were returned as three numeric values in a multi-value list.  What if we could use our new alternate syntax and say 

@DbColumn("Midas"; @DbName; @DocumentUniqueID; "Body";
                           2; "Format='Currency' SkipTitle='Yes' "; 625:775:375);

and the table would change to:

IBM Value Package for Developers
Annual fee
North America, Europe, Middle East, Africa 
Latin America 
Asia Pacific region

Now try to guess what would happen if we specified the following two commands in a row with too many values:

@DbColumn("Midas"; @DbName; @DocumentUniqueID; "Body";
                           2; "Format='Currency' SkipTitle='Yes' "; 625:775:375:515:775);
@DbColumn("Midas"; @DbName; @DocumentUniqueID; "Body";
                           2; "Format='Currency' SkipTitle='Yes' ";
                           "North America, Europe, Middle East, Africa":"Latin America":"Asia Pacific region":
                           "Antarctica":"Pacific Islands");

You would get the following result:

IBM Value Package for Developers
Annual fee
North America, Europe, Middle East, Africa 
Latin America 
Asia Pacific region
Pacific Islands

There are a few loose ends, but this is nearly complete for @Midas Formulas.  Let me know what you think.

Copyright 2003 Genii Software Ltd.

Thu 18 Sep 2003, 10:02 AM
As a follow up to yesterday's post on @DbColumn/@DbLookup on rich text tables, one of the questions I get about our beta @Midas Formulas is "Where would you use it?"  While there are lots of answers, one that might not be obvious is "From Java".

Because Java offers the ability to evaluate formulas, it can call an @Midas formula and either get the results or simply make the change.  No configuration or special linking or translation layers of JNI.  If the @Midas software and license are on your server or client, Java can use them.  This works in ND6, R5, and might even in R4.6, although I can't recall whether the evaluate method was available from Java then.

So why would this matter?  What might you want to do from Java with rich text.  [NB: Lotus clearly doesn't think you want to do much, as their rich text manipulation from Java is seriously lagging even the small amount of rich text manipulation they offer from LotusScript.  But I digress...]

Three simple examples come to mind.  The first is based on yesterday's post.  If you have a rich text table and want to look up a column of data, you could use code such as the following:

Vector v=session.evaluate("@DbColumn(\"Midas\":\"NoCache\"; \"\"; @DocumentUniqueID; \"Body\"; 2; \"Format='Currency' SkipTitle='Yes' \");", doc);

This would give you a vector array of numeric values you could then use as you needed.  Pretty simple, and no other configuration or setup required.  Escaping the quotes is a bit annoying, but other than that, it is a lot easier even than getting the same information using the Midas Rich Text LSX, where you would have to connect, define the table chunk, loop through the rows, etc.

The second sample uses a different feature of @Midas, which is the ability to convert rich text to HTML.  With the following simple code snippet, you could retrieve the HTML for the content inside a particular table cell in a rich text field.  This HTML fragment and others like it could then be used to build a webpage dynamically based on choices made in the Java code.

Vector v=session.evaluate("@DbCommand(\"Midas\":\"NoCache\"; \"Connect\"; \"\"; @NoteID; \"Body\"; \"GenerateHTML\"; \"Table 2; Row 2; Column 3\"; \"HTML\"; \"Generation=Fragment MoveStylesInLine=yes ImageRelativeToPage=yes \")", doc);

The third example uses search and replace to modify the form stored in a document, thus allowing the form to be dynamic:

Vector v=session.evaluate("@DbCommand(\"Midas\":\"NoCache\"; \"Connect\"; \"\"; @NoteID; \"$Body\"; \"ReplaceText\"; \"Dollars\"; \"Euros\"; "Text"; ; \"Save\")", doc);

So, whether Java is your cup of tea or not, you can see how relatively easy it would be to add rich text functionality to a Java agent by add a little simple @Midas code.  If you would like to sign up for the beta test, we are planning on sending out a new beta release shortly.  If you are already signed up, even if you haven't received a beta yet, you will be contacted automatically.

Copyright 2003 Genii Software Ltd.

Wed 17 Sep 2003, 11:20 PM
I thought I'd focus on a few of the things we are trying out with our new @Midas Formulas, which is beta.  If you would like to sign up for the beta test, we are planning on sending out a new beta release shortly.  If you are already signed up, even if you haven't received a beta yet, you will be contacted automatically.

@DbColumn/@DbLookup on rich text tables
One of the really interesting features is the ability to treat rich text tables as data, a mantra some of you who have seen me speak at Lotusphere may have heard before.  Why shouldn't you be able to retrieve data from tables in rich text fields?  Why shouldn't you use commands most Notes developers are very familiar with for retrieving that data?

Well, with @Midas Formulas, you can do this.  For example, if a table such as this one from the Partnerworld site were in a rich text field called Body on the current document:
IBM Value Package for DevelopersAnnual fee
North America, Europe, Middle East, Africa $595 
Latin America $750 
Asia Pacific region$350

you could specify a command such as:

prices := @DbColumn("Midas"; @DbName; @DocumentUniqueID; "Body"; 2; "Format='Currency' SkipTitle='Yes' ");

and prices would be three numeric values in a multi-value list.
You could also specify a formula such as:

price := @DbLookup("Midas"; ""; @DocumentUniqueID; "Body"; "Latin America"; 2);

and price would be a string "$750"

Pretty cool, eh?  You can do this to avoid using separate documents for rows of data.  What is particularly cool is that exactly the same lookups would work with exactly the same results if your table looked like this:
IBM Value Package for DevelopersAnnual fee
North America, Europe, Middle East, Africa $595 
Latin America $750 
Asia Pacific region$350

Even the use of background colors, text attributes and graphics, which help make the data more presentable when the rich text field is displayed, do not prevent use of @DbColumn and @DbLookup.  So, does that make you think maybe you would like to take a look at @Midas Formulas?

Copyright 2003 Genii Software Ltd.

Tue 16 Sep 2003, 04:48 PM
Caveat: this blog entry has nothing whatsoever to do with Derek and the Slammers, or even with Elvis.  Mea culpa!  However, I still feel that the title is apt, as you will see.
It is an open secret that I am working on a RT -> HTML -> RT solution that should be out in beta soon.  As I get closer to an actual working product, I am faced with many technical hurdles (pah, I spit in the face of technical hurdles) and a few questions that are not technical.  They are social, perhaps even cultural.  These questions have to do with whose world view is "correct".  When the web client and Notes client coexist, should web norms be followed or Notes norms or should each follow its own norms.  This is similar to the question of whether Notes should look the same on the Mac as it does in Windows.  If the Mac UI guidelines are followed, it will be good for Mac users, and confusing for those who UI is different when they visit another machine.  If the Windows UI guidelines are followed, it will have the opposite effect.
To make matters worse, the web client is highly driven by CSS these days, which means that all sorts of UI decisions may vary by user/implementer/page.  Of course, UI issues are not the only ones that matter.  Let's look at a simple example where the differences show up clearly:
  <tr><td>Look, Mom!  I'm famous.  I'm on a web page</td></tr>
  <tr><td>That's nice, dear.  Now wash your hands and come in for dinner.</td></tr>
  <tr><td>Honey!  I'm home!</td></tr>
Basic HTML Table
This is a table described in HTML.  It looks like this (in IE 5.5 at least):
Basic Notes Table
Now, let's look at the same basic idea in Notes. A table created with default values with four rows with the same content except set as Fixed width because I needed it to fit on this narrow blog entry.
What are the differences?  Well, there is no border on the HTML table, but there is a border in Notes.  The font size is larger (although that varies by browser settings as well).  There is no blank row shown when the HTML row is blank, but there is a blank row shown in Notes.
Basic Notes Table rendered by Domino
Now, look at how Domino shows the Notes table on the web:
Different again!  The borders show, as it could be argued that they should, but as some sort of extruded lines.  The blank row shows, but not at the same height as the others,
Basic HTML table rendered by Notes
Finally, look at how Notes imports the HTML table:
That is not too different, but the blank row is included.
Who could draw a conclusion from this?  There are just more questions raised.  With good Notes/Web coexistence, should the original table be the guide, so that an HTML table created without borders and rows should be imported that way into Notes?  Should a table created in Notes have the thin lines as it does in Notes when it is shown on the web?  Should CSS be allowed to change the default appearance of a table even if it was created in Notes originally?  Should CSS change the table as it appears in Notes if it was created on the web using that particular CSS?  For that matter, should you be able to change the CSS and alter the appearance in Notes?
All of these are possible, but it is not clear which are correct or preferable.  After all, who died and made me Elvis?

Copyright 2003 Genii Software Ltd.