Notes / Domino by Daniel Nashed 

Nomad Web server connection options  

By Daniel Nashed | 1/25/23 7:30 AM | Infrastructure - Notes / Domino | Added by Oliver Busse

Nomad Web is a modern HCL client offering in form of a Progressive Web Application (PWA) running in your web browser. In addition to Windows or Mac, it also works on Ubuntu and other Linux distributions! So there is finally a client offering for Linux clients again! The Nomad Web application is installed on a server providing the required files for download. Those files can be stored on a SafeLinx or Domino/Nomad Web server.

Windows Sandbox - A feature you should know  

By Daniel Nashed | 1/23/23 2:20 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

The sandbox can be a very useful tool for many different situations. I am often using it for Domino server or client install tests. But there are many other use cases including training environments etc. It's a full throw away sandbox environment recreated every time you start it. The only limitation is that you can't reboot the Windows for example after a software update. But even installing the Windows re-distributable run-time package does not require a boot. Most applications like Notes/Domino install it on their own. I needed it to test my own applications. But there is an easy way to download and silent install it:

NGINX TCP Stream with SNI support. More than helpful for lab environments  

By Daniel Nashed | 1/23/23 2:15 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

In production you usually want centralized certificate handling and off-loading TLS termination to a load-balancer. I posted scripts to have NGINX realod certs automatically from Domino CertMgr via HTTPS to leverage Domino's Let's Encrypt implementation. But sometimes you really want all your servers directly exposed over TLS. For example in a lab environment with limited resources and only one IP, you might want to still have each of the hosts expose their services on their own.

How to get the error message for a Notes error code  

By Daniel Nashed | 12/13/22 1:34 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

Sometimes scripts or Domino server commands only return an error code and you would like to know the error message. There is an easy way to get the error message back from a server command. "show message [module] In most cases you don't need server tasks specific error messages and just use the decimal error code.

Restic – Command Line Tool supporting Windows VSS  

By Daniel Nashed | 11/24/22 2:03 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

Now that Domino 12.0.2 is has a native VSS Writer, we can look into new interesting integrations. In my session at SUTOL conference this week, I showed a first version of a Restic integration for Domino 12.0.2 via VSS. Restic is a very interesting application (https://restic.net) It's a single binary written in GO And uses a approach like Borg Backup uses. But in contrast to Borg Backup it has full Windows support. This includes VSS Writer + AutoRecovery support! It is Open Source, efficient, flexible & secure. And very simple to setup & use!

Docker cp with permissions and owner change  

By Daniel Nashed | 10/11/22 2:22 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

By the default the ownership of a file copied into a running container is always root:root. Depending on how you want to use the copied file, this ownership isn't what you want. You can't pass user/group or permissions to the docker cp command. Changing the owner or mode would need root permissions inside the container. Containers usually run with an unprivileged application user. For Domino this is notes:notes with the IDs 1000:1000.

Domino 12.0.2 One Touch setup with Let’s Encrypt certificates  

By Daniel Nashed | 9/15/22 3:32 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

One Touch setup (OTS) is quite powerful tool. But sometimes you need to know exactly what happens and combine functionality to make best use of it. With Domino 12.0.2 OTS creates certstore.nsf automatically and you can let it create a MicroCA for you. But what if you want to use a Let's Encrypt certificate instead? There is a quite simple way to just find and update the existing document with a appConfiguration. And if you specify notes.ini CertMgr_ACCEPT_TOU=1 the ACME account license agreement will be automatically accepted (already part of 12.0.0).

Download certificate chain without OpenSSL  

By Daniel Nashed | 9/12/22 3:17 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

Usually OpenSSL is the tool of choice for all type of certificate operations. But what if no OpenSSL command line is available? Like in a Domino container where you can't install software? After some research, I came up with the keytool, which is part of the JVM Domino ships.

K3s, Podman and a registry  

By Daniel Nashed | 9/5/22 2:03 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

Rancher Desktop is a great all-in-one desktop environment. When running it with the Docker back-end you have all in one environment for development and run-time. For a server, K3s (https://k3s.io) is my platform of choice. It is production ready and easy to deploy. For Kubernetes, you always need a registry to pull images. As soon you need custom images, you will need a registry to upload and download your image. K3s allows you to configure private registries. You could use any registry. I am just running the registry Docker image on Podman in my environment.

Domino Community Image - New Nomad Server install option  

By Daniel Nashed | 7/30/22 10:22 AM | Infrastructure - Notes / Domino | Added by Oliver Busse

The Nomad server is a new offering to directly add Nomad support to your Domino server instead of using a SafeLinx server. Recently I added a SafeLinx container to the Domino community project. Now I am adding the Nomad Server to the Domino image as a new build options.

Domino Container automation testing  

By Daniel Nashed | 7/25/22 12:20 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

Containers are not only a good way to run Domino. It is also the perfect environment for automation testing. Domino 12 introduced OneTouch Setup to automate deployments, which also lets you create reproducible Domino test server scenarios. As a starting point I am building an automation test for the Domino image itself. So in future for every commit on the Git repository I can run automation to ensure the image works. The test automation can be used in your own environment as well after an image built in your environment. It can be also be extended for your own application testing.

Customizing Domino Backup mail notifications  

By Daniel Nashed | 7/22/22 6:42 PM | Infrastructure - Notes / Domino | Added by Oliver Busse

Domino Backup offers to send e-mails depending on the status of your backup. By default you are getting an e-mail in case of error or warning. I am rarely getting error messages from my servers. In this case here I updated my server to a new kernel and ZFS drivers failed to build.

Nomad Server 12.0.2 on Linux just works  

By Daniel Nashed | 7/20/22 10:32 PM | Infrastructure - Notes / Domino | Added by Oliver Busse

The Nomad Server is a small component, you install on your Domino server. And it is bundled with the Nomad Web files. So it is a all-in one server add-on solution. The installation sounds more complicated then it is. It's really simple to install. And I am thinking about making it an install option for the Domino community container image.

Why run Domino in a container today  

By Daniel Nashed | 7/20/22 2:03 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

As my of you know, I am a big fan of running Domino and other applications in a container. This can be a classical Docker/Podman deployment or K8s. Containers might not be good for everyone. But a lot of software is available in a "Docker image", which can run in multiple environments. Domino's main deployment model will not change to Docker.

Domino 12 Restore point in time  

By Daniel Nashed | 7/15/22 1:23 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

Domino Backup can restore databases point in time! For other backup applications this functionality is usually only available with archive transaction log. But with circular translog or linear translog mode, the most current backup should have all the translogs available to recover point in time as well. Domino Restore allows you to restore point in time in that case. Even it is not guaranteed that the translogs are still there, this can be still a good configuration if you have servers without dramatic load.

Linux shell scripts: Difference between "set" and "env" -- fixed the Domino start script  

By Daniel Nashed | 7/11/22 4:28 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

The Domino start script has always been using "set" to list the environment variables, before running the sever. This information can be important to understand the environment passed to your Domino server at start-up. It turns out that there had been a change over time, which causes much more information to be listed, then just the environment variables.

Domino ZFS Snapshot Backup  

By Daniel Nashed | 6/20/22 2:05 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

ZFS is one of my favorite file-systems. And I posted before about using it as a backup target. The integration is pretty simple with Domino backup, because it is a simple file backup. Now that we have the new VSS Writer for Domino 12.0.2 on Windows, it is time to look into ZFS snapshots.

Recovering a lost Domino server notes.ini quickly  

By Daniel Nashed | 5/26/22 1:39 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

On Linux by default the notes.ini is in the data directory. On Windows it is per default in the binary directory. You could move it to the data directory, which would make sense from backup point of view in many cases anyway. But what if you have it in the program directory and install a new major version where you get rid of all your binaries as a best practice?

SHA512 is faster then SHA256  

By Daniel Nashed | 5/13/22 6:12 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

Quite interesting results.. I have been looking into different hash algorithms to see the overhead today. It turns out that SHA256 is the slowest option and SHA1 is the winner. But it is interesting, that SHA384/SHA512 are also faster then SHA256

Faster Domino server restart   

By Daniel Nashed | 5/13/22 6:11 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

Domino waits for 10 seconds after shutdown before are restart for some legacy reason. There is a notes.ini variable to reduce the number of seconds. I tested with Domino 12.0.x that it can be reduced to 1 second.

K8s Certificate Manager with Let’s Encrypt  

By Daniel Nashed | 5/5/22 4:41 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

Domino certificate manager works like a charm and is the best option for native Domino 12 certificate management. But in a K8s environment you might want to better have certificates deployed outside Domino in front of your Domino K8s service. Mostly you will use a so called Ingress controller, which offloads your TLS traffic. I took a look into https://cert-manager.io/docs/concepts/certificate last night. It turned out the issues I ran into only occurred because of a messed up k3s installation. After I re-created my server, I was ready to go in minutes.

Full Domino Fail2Ban Integration  

By Daniel Nashed | 3/21/22 3:45 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

This week we tried to get Domino 12.0.1 IP based blocking working for our DNUG server. It turns out to work great for the HTTP protocol. But our Sametime server got blocked on port LDAPS when verifying log-in information via LDAPS.

HCL Domino Docker Container - Moved to a new home!  

By Daniel Nashed | 3/16/22 3:25 PM | Infrastructure - Notes / Domino | Added by Oliver Busse

The Domino Docker project was started by Thomas Hampel with Domino 9.0.1 at IBM. He introduced me to the project to contribute my Domino start script. I wrote a lot of code and added a lot of functionality since then. Thomas and I did many presentations together and it always was and will continue to be one of our favorite projects!

SpamGeek works -- O365 finally got blocked sending too much spam  

By Daniel Nashed | 3/15/22 5:29 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

Ohhh ... I was wondering that some mails did not reach me any more. And I took a look into my SpamGeek logs. Over time the SPAM score of the domain got higher and now it was blocking also other senders from that domain. It got higher because the words I added to the negative list. And the ratio between good and bad mail changed..

Leveraging Docker Registry for your own deployments  

By Daniel Nashed | 3/7/22 12:52 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

Sometimes you don't see the most straightforward even it is right in front of you... I have always used my own registry for deploying Docker images. Either a very light registry2 ( https://hub.docker.com/_/registry ). Or a harbor registry ( https://goharbor.io/ ). The harbor registry is very nice! But requires a separate server. The own small Docker registry requires a TLS certificate and unless you switch it to port 5000, it might block your HTTPS port -- unless you put NGINX etc in front and use SNI etc ..

Domino Start Script official GitHub repository released  

By Daniel Nashed | 2/28/22 2:22 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

The start script has quite some history and is longer around than Domino on Linux. Most of you probably don't know that I wrote it originally for HP-UX and then extended it with Sun Solaris and AIX support (that makes me feel a bit old ..). When Domino on Linux was introduced in V5.0.3, I added Linux support. And later we had to move from init.d to system.d services to start the server. It has always been free and when I contributed it to the Docker Community image I changed the license to Apache 2.0 open source license. But there was no own public GitHub repository and the official channel was still to request it by mail. Now I am moving the start_script source directory from Domino Docker GitHub repository to it's own home.

New technote about Domino 12.0.1 DAOS hang  

By Daniel Nashed | 2/22/22 2:17 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

It turned out the hang situation which can occur in combination with DAOS on larger servers is not completely fixed in 12.0.1 IF1. In case you are running into hangs check the brand new technote for details: https://support.hcltechsw.com/csm?id=kb_article&sysparm_article=KB0096771 The issue behind the problem is complex and the HCL team is working an a solution. In this situation you should know your most reasonable options

Important Domino 12.0.1 IF1 for customers using DAOS  

By Daniel Nashed | 1/10/22 2:50 AM | Infrastructure - Notes / Domino | Added by Oliver Busse

This issue has been already in 12.0, but was discovered to late to be included into 12.0.1. HCL worked hard to get IF1 out ASAP. There was a hotfix already available end of the year. But IF1 take a bit longer than distributing a hofix -- there is more testing is involved.

Introducing Domino One-Touch JSON templating - Without manual JSON editing :-)  

By Daniel Nashed | 12/30/21 6:09 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

Domino One-Touch setup with JSON format is really great stuff -- I love it since day one. But it might be a bit difficult to edit for many admins -- Even if you use an prepared template, you have to edit the JSON file in an editor to specify server name, etc. Using variables ala Helm or Ansible makes a lot of sense and if you leverage existing JSON config templates, you might get away with not editing JSON at all. Both using the {{ Variable }} syntax. But I am more used to the shell variable syntax: ${Variable}. So I implemented both.

Domino 12.0.1 One-Touch setup supports MicroCA and import existing certs  

By Daniel Nashed | 12/30/21 2:28 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

Did you notice already that the One-Touch setup supports importing TLS Credentials for a first server setup? You can pass a *.kyr file, a PKCS#12 (*.p12, *.pfx) or *.pem file. The files can even have a password and you can mark the resulting TLS Credentials file for export with a new password! So the full import functionality added in Domino 12.0.1 CertMgr UI is exposed in One-Touch setup for ENV variable and JSON formatted setup!