Notes / Domino by Daniel Nashed 

Domino Container Environment Install Script  

By Daniel Nashed | 7/2/25, 4:31 PM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

It has been a round for a while. I am too lazy to do the same installations over and over again manually. In addition it helps me to get customer environments all setup in the same way. This script clones the Domino container project and Domino start script. It also installs Docker from the official repository along with the Domino download script and also the container control script. This is the fastest way to setup a container environment.

Engage session follow-up – Domino 14.5 AutoUpdate downloads  

By Daniel Nashed | 5/26/25, 8:48 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

Thanks to everyone who attended my 8:00 AM session on Wednesday. One topic raised during the session deserves a closer look: how Domino AutoUpdate retrieves installation artifacts. To download product.jwt, software.jwt, and the Notes/Domino web kits, you need at least one server with outbound connectivity to My HCL Software portal (MHS) and the HCL Domino fixlist servers. Domino AutoUpdate a supports HTTP proxy configurations, including authenticated proxies, which should work in most enterprise network environments. All downloads are validated against the software.jwt, which includes signed metadata for all supported software packages. This model fits most connected environments. Completely air-gapped setups are uncommon, and to date, there haven’t been strong or clearly defined requirements for full offline AutoUpdate workflows. However, it’s still possible to override download URLs in AutoUpdate documents to manually provide software.jwt and web kits from internal sources.

Configure an addtional Notes port on a server  

By Daniel Nashed | 5/15/25, 6:48 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

The previous blog post was more dealing with the background about having a second Notes TCP/IP port. This post focuses to setup a new Notes port end to end using the DNUG Lab environment as an example. The server I am configuring has two separate IP addresses on two different network cards. But the same procedure would also work with IP addresses in the same network.

Benefits of running domino with multiple TCP/IP ports  

By Daniel Nashed | 5/15/25, 6:47 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

Support for multiple TCP/IP ports has been part of HCL Domino since the early days. Back then, it was first essential to support multiple simultaneous modem connections. It also proved valuable for clustered servers using dedicated network cards. While today’s networks offer 1 Gbit/s or even 10 Gbit/s speeds—making multiple ports less necessary from a raw bandwidth perspective—there are still compelling reasons to use multiple Notes ports in modern environments.

Adding trusted roots to Domino containers  

By Daniel Nashed | 4/7/25, 4:03 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

Linux and Domino comes with a good set of public trusted certs. But in corporate environment you often have to add your own trusted root for a corporate CA. This starts with Linux which needs certificates to validate repository servers and other resources. Domino trusted roots But also within Domino there are are trust stores which need might need central management. Domino Directory Trusted roots, certstore.nsf Trusted roots can be easily centrally updated. But the following two trust stores are more difficult to manage: /local/notesdata/cacert.pem used for HTTP Requests in Lotus Script and other backend code using curl Domino JVM trust store used by Java

What you should know about Domino "res" files on Linux and AIX  

By Daniel Nashed | 4/3/25, 4:56 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

res files actually come from Windows and are used to translate strings for UI and errors. Those res files are usually linked to the Windows binary. Linux and AIX also use "res" files in a res/ directory below the binary directory. The files are essential for a server. All the core code string resources are in strings.res. Most Domino native servertasks also use string resources.

Wolfi OS - Secure base layer for containers  

By Daniel Nashed | 4/3/25, 4:54 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

This project is pretty cool. It's a container only OS using the kernel from the host. But it has a couple of really interesting design goals. https://github.com/wolfi-dev/ They build container base images with the minimum number of packages and "CVE free" as much as possible. So their own containers for NGINX for example really only have NGINX and nothing around it -- not even a shell unless you install a :latest-dev container.

How to update Domino when running in a container?  

By Daniel Nashed | 4/3/25, 4:52 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

Today I just updated my production environment to Domino 14.0 FP4. Let me show you how it works if you have everything setup.

How to add a trusted root to Linux  

By Daniel Nashed | 2/3/25, 2:57 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

I am still adding custom trusted root support to the Domino container project. You will be able to just specify a trusted root to add to the local Linux trust store. Like other low level functionality this works differently on different Linux flavors. Here is what I am adding for SUSE, for Debian/Ubuntu and basically all the other Redhat/RPM based systems (I rested Rocky, Alma & Co so far).

DBMT tool enhancements in Domino 14.5 EA2   

By Daniel Nashed | 1/7/25, 2:39 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

Sometimes small changes open many new possibilities. The following DBMT tool command line options are added to DBMT in Domino 14.5 EA2: -systemDbs (-sd for short) Allows compact to process system dbs, which are usually ignored), as well as databases listed in the dbmt_compact_filter.ind file. -regex (-re for short) Now a database name can be specified using regular expressions. If an .ind file is specified, the database names listed in the .ind file can be regular expressions. -validateDbs (-vd for short) Does not execute the updall or compacts, but outputs the list of databases that could be affected by the DBMT command (mainly to validate -regex inputs). Can be used in combination with -sd

Notes Timedate explained  

By Daniel Nashed | 1/7/25, 2:38 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

There have been a couple of partner blog posts speculating about the background of the recent Domino 13.12.2024 problem, which might be a bit misleading. For the background of what happened in detail and how HCL addressed the problem please wait for the official technote update. But what I can tell is that HCL fixed it on a lower level function addressing all functionality in Domino and business partner applications using the effected functionality. This means the only safe way is to apply the Interim fix provided by HCL for all supported releases including the extended support versions! What I also can state is that all Notes TIMEDATE functionality is working as intended and are designed to handle date times from 1.1.1 to the end of all times.

New project Domino Download Server  

By Daniel Nashed | 12/30/24, 7:11 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

Over the x-mas I had a bit of time to work on an idea I had already a while ago. Some customers can't directly connect to the internet. Not even with a proxy. Domino AutoUpdate and also the Domino Download script both support proxy environments including authenticated proxies. The Domino Download script leverages the curl command-line which is very flexible. But also Domino AutoUpdate has full proxy support. Still some environments can' download anything from the internet. Some are even air gapped. The idea was to come up with a NGINX based service which could be the source for all your Notes/Domino downloads. I wanted it to work in different environments.

Notes intermittently hangs or opens mail or other database slowly after 30 minutes of inactivity  

By Daniel Nashed | 10/28/24, 2:20 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

Notes intermittently hangs or opens mail or other database slowly after 30 minutes of inactivity This might help you in some network situations and it came up today in the OpenNTF Discord chat. TCP/IP keep alive is a functionality in the network stack to tell the server's TCP/IP stack and also the active components like firewalls, VPNs etc, that your session is still alive -- even the application is not sending any data. The Windows default keep interval is 2 hours. This Windows sends a keep alive for a TCP/IP session only. Linux and MacOS have a default keep alive interval of 75 seconds, which is a much more reasonable default. On Windows you can change the value by adding a new registry value, specifying a shorter keep alive interval in milliseconds. A good default value would be 75 seconds like on Linux and MacOS.

Key Rollover vs Certifier rollover  

By Daniel Nashed | 10/28/24, 2:18 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

This is probably a topic many admins never really looked into and you might still run with your very old 630 key size. Key size and certificate key size play an important role in your security and you should be aware of it. Key Rollover Rolling over keys is a quite normal operation. It's a best practice to rotate keys at least when the recommended key strength changed. Rolling over a key is client side initiated but requires an admin action. Certifier Rollover When rolling over certifiers you are creating a new key for your certifier and sign it with the right signing ID. For your organization certifier this will be the organization certifier itself which signs itself. Once that operation completes you have to re-sign all OU certifiers, server IDs and Notes.IDs step by step in this order. You also have to take care of all cross certificates, Vault trust certificates. The process is quite complex and needs planning:

Check the minimum client version for your Notes application  

By Daniel Nashed | 10/25/24, 3:12 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

Notes provides new functionality in Lotus Script and there also Java classes added to the client. Lotus Script Named documents have been introduced in Notes/Domino 12.0.1. I have just written an application which needs a Java class which is introduced in Notes 12.0.2 as it turned out. So I came up with a simple check I am going to add to all my applications which use more current functionality. You can drop this code into the PostOpen script of any database and switch to the right constant

Domino Container image custom add-on support enhancements  

By Daniel Nashed | 10/14/24, 3:19 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

There is a custom add-on functionality Martijn and Roberto just blogged about this week. https://blog.martdj.nl/2024/10/10/building-custom-add-ons-for-your-domino-container-image/ https://www.robertoboccadoro.com/2024/10/10/upgrading-ontime-in-a-container/ This was the missing trigger for me to look into it again. It's a quite new functionality which wasn't fully documented yet. Documentation I have added a new documentation mark down page-->https://opensource.hcltechsw.com/domino-container/concept_custom_addons/

Linux LSOF is causing 100% CPU load inside a container in some configurations  

By Daniel Nashed | 10/2/24, 4:34 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

Linux LSOF is causing 100% CPU load inside a container in some configurations https://blog.nashcom.de/nashcomblog.nsf/dx/ https://blog.nashcom.de/nashcomblog.nsf/feed.rss RSS - Daniel Nashed's Blog Daniel Nashed's Blog Daniel Nashed Linux LSOF is causing 100% CPU load inside a container in some configurations Linux Domino Container width=device-width, initial-scale=1.0, minimum-scale=1.0 Daniel Nashed's Blog ../nashcom.css ../dx/imprint.htm Imprint Domino on Linux/Unix, Troubleshooting, Best Practices, Tips and more ... Search Search Search Search alt Daniel Nashed # Tags Tag: 64Bit ../archive?openview&title=64Bit&type=cat&cat=64Bit 64Bit Tag: ACME ../archive?openview&title=ACME&type=cat&cat=ACME ACME Tag: ACME HTTP-01 ../archive?openview&title=ACME%20HTTP-01&type=cat&cat=ACME%20HTTP-01 ACME HTTP-01 Tag: ADFS ../archive?openview&title=ADFS&type=cat&cat=ADFS ADFS Tag: AdminCentral ../archive?openview&title=AdminCentral&type=cat&cat=AdminCentral AdminCentral Tag: AIX ../archive?openvie

Disabling XPages if not needed reduces open files and HTTP start/stop time  

By Daniel Nashed | 9/30/24, 4:30 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

While working on setup automation I often ran into HTTP startup challenges. It can take up to 40-50 seconds until the HTTP task is started. If you look at open files, you notice that each thread has more than 70 files open. This sums up to up quite some files and the HTTP server start/stop time is much slower. In case you don't use XPages there is a simple switch to disable the XPages run-time and only load the standard Java components. notes.ini INotesDisableXPageCMD=1 I first had the impression Java in general would cause overhead on start. But my tests drilled down to XPages/OSGI.

Domino 14.0 FP2 IF1 installer might fail on new machines -- VCRUNTIME140 32bit is missing  

By Daniel Nashed | 9/24/24, 1:06 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

I ran into this today when testing and got a customer reporting this one hour later. So it was easy to reply with a root cause and solution. Domino is a 64bit application. Therefore the Windows run-time installed with the Domino release installer is 64bit only. The Fixpack installer has no VC runtime requirements. But it turns out the hotfix installer, which is also used for interim fixes is also a 32bit installer and has VC dependencies.

Domino does not shutdown cleanly when Windows is rebooted or shutdown  

By Daniel Nashed | 9/11/24, 6:23 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

When stopping the Domino service manually, the Windows service control manager (SCM) waits sufficient time to shutdown Domino cleanly. But it turns out a Windows shutdown or reboot does not wait sufficient time for service termination. This is critical because it would kill running Domino processes without notice. Even with transaction log enabled, this isn't a desirable situation.

How to find out what is eating my disk space on Linux?  

By Daniel Nashed | 9/11/24, 6:22 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

If you don't know the Linux tool ncdu, this will make your day. The tool by default scans from where you are or any directory you specify. Specially when running on WSL you might want to use excludes. On top there is a delete option, which can be quite helpful when you find large files you don't need. I am using it for years and it did safe my IT life more than once. And it is very fast...

Domino One Touch Setup (OTS) advanced examples and helpers  

By Daniel Nashed | 7/29/24, 3:22 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

OTS is a very powerful and flexible feature of Domino 12+ which has been extended in each dot release since then. I am OTS a lot in the container world. But it also works on Windows. It perfectly fits into the container world. And we added a couple of integration points into the container image. Because I got a couple of questions I wrote up some examples, related information and also an Lotus Script agent to extend the functionality. The agent is intended to be an example how to wrote own integrations and also to leverage and extend the existing agent for own needs.

Pretty-Printing JSON in Notes Client and Domino  

By Daniel Nashed | 7/29/24, 3:20 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

The Lotus Script class for reading and writing JSON is that easy. There are not many examples and some functionality is missing. JSON can be either condensed without any new lines and indentation. That's great when you use it for back-end processing or REST services. Why is pretty printing important But in some cases you need pretty formatted JSON. Specially when you want to maintain it manually and extend it. For example for Domino OTS JSON files :-) When you use JSON based configuration pretty printed JSON is very helpful. Condensed JSON is also difficult to check into Git. Everything looks modified when it is a single line.

HCL Nomad server 1.0.12 IF1 shipped with same file name than 1.0.12  

By Daniel Nashed | 7/22/24, 6:13 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

Nomad 1.0.12 has been replaced with a 1.0.2 IF1 version. MHS has only the new version. The old version can't be downloaded any more. But they left the file names the same. So you can't distinct the files by name once you downloaded them. So you have to delete the old file and re-download it. The same file name with a different content (resulting in a different hash and size), breaks automation. For example it broke the Domino container build automation.

Running Domino Windows container image on Windows 2022  

By Daniel Nashed | 7/8/24, 1:43 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

Two years ago I have been looking into Domino in a Windows container already. The main purpose was to understand the technology and if this makes sense to be used in general. IMHO container technology is mainly helpful on Linux. Containers on Linux use core OS level functionality, which is part of the Linux kernel. Only Linux makes sense for production use for me. But a Windows container can be a great test environment for automation testing and other test use cases. I revisited my container build on Windows this weekend and first updated it to Domino 14 and also updated all involved tooling like 7Zip. In addition I looked into how I could leverage a Windows container image for testing.

New Nomad Server features -- ACME HTTP-01 challenge support & HTTP redirects via port 9080  

By Daniel Nashed | 7/8/24, 1:42 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

There are two new features in the latest Nomad Server versions, introduced to Nomad Server without big notice. I just got the question from a partner why Nomad Server now binds port 9080 in addition to port 9443 and the internal communication port (only loop back). The port might be used by other applications like the IBM Spectrum Protect (TDP) -- which was the problem in this customer case. It turns out the TDP Java based restore GUI and does not work in combination without changing or disabling the port.

Building applications in a build container  

By Daniel Nashed | 5/13/24, 4:32 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

Specially when developing for different target versions of an OS or an application a build container can be very helpful. But build containers are also really helpful in larger teams when everyone should use the exact same build environment. The Domino container project supports adding the Notes/Domino C-API SDK to the container image. In case of Domino libnotes.so is required. Therefore compiling requires at least an installed Domino server with the same or newer version than the SDK version. I built a Domino 14.0 FP1 image including C-API 14.0 and tagged it hclcom/domino:build. For this blog post I am using the simple test program in the container projects automation test directory --> https://github.com/HCL-TECH-SOFTWARE/domino-container/tree/main/testing The directory is defined as a volume inside the container /build.

Domino Container Project: software.txt link & new start script version  

By Daniel Nashed | 5/9/24, 3:22 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

The container project contains a software.txt file with all the web-kits used to build images. For some technical reasons the file was located in two places. 1. The build.sh script uses software.txt for checking web-kits before the image build starts. 2. The actually image build process uses software.txt to verify the downloaded web-kits before installing them. software.txt and current_version.txt can also be added to a custom software directory (SOFTWARE_DIR) or remote download location (DOWNLOAD_FROM).

HCL SafeLinx 1.4.2 available -- New best friend "Domino CertMgr"  

By Daniel Nashed | 5/2/24, 9:21 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

This was a quite high rated AHA idea. So the SafeLinx team and Domino team teamed up to implement it. The flow is integrated into the SafeLinx UI and SafeLinx also allows ACME challenge "passthru". There isn't any change in CertMgr needed. It is implemented in a way that you could implement your own integration flows. If you have a specific integration idea, ping me. I can point you to the right direction.

Updating autoupdate.nsf with the new template (14.0 08.03.2024)  

By Daniel Nashed | 4/17/24, 3:56 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

The new fit & finish work and the new autcat.nsf integration requires template changes. Please make sure you are getting the template version 14.0 from 08.03.2024 and not the earlier version from 03.11.2023 shipped with Domino 14. When deploying the container image I noticed an issue with the folder permissions where the container image is getting template updates for Fixpacks. The directory /opt/hcl/domino/notes/latest/linux/data1_bck/140FP1/localnotesdata contains updated templates. But the directory can be only accessed by "root" and the container runs with the "notes" user. This is not new to 14.0 FP1. Also 12.0.2 fixpacks had the same permissions, but nobody noticed the missing updates.