This is the documentation of an old opsi version. The documentation for the current version 4.3 can be found here.

opsi WAN/VPN extension

The WAN/VPN extension module allows to integrate clients, that are connected via low speed connections. This chapter is about configuring and maintaining the opsi WAN/VPN extension.

Preconditions for using the WAN/VPN extension

This opsi extension is currently in the cofunding process and not free. For more details see opsi Extensions.

There are some preconditions to use the WAN/VPN extension module. The feature 'Produktgruppen' is required, which is available with opsi 4.0 and above. Also the packets 'opsi-client-agent' and 'opsi-configed' are required, which come with version 4.0.1.

At the moment, the simultaneous use of both "WAN extension" and "installation on shutdown extension" is not supportet. On the same opsi server with different clients, these opsi extensions can be used.

Table 1. Required packets
opsi-Packet version

opsi-client-agent

>=4.0.1-1

opsi-winst

>=4.10.8.12

python-opsi

>=4.0.1-7

opsi-configed

>=4.0.1.6-1

General overview of the WAN/VPN extension

opsi software deployment is mainly doing the following steps:

  • The 'opsi-Loginblocker' at client system startup prevents the users from logging on.

  • The 'opsiclientd' service running on the client connects the 'opsi-configserver'.

  • If any 'Produktaktionen' are set for the client, it mounts a share from the 'opsi-depot'.

  • The 'opsi-winst' is starting and also connects to the 'opsi-configserver'.

  • The 'opsi-winst' executes the 'Produktaktionen', using the share from the 'opsi-depot'.

  • If a reboot is required, it executes and the process starts all over.

  • When all the 'Produktaktionen' are completed, the log files are transferred to the 'opsi-configserver' and the user logon is unblocked.

Now we will look at the special circumstances of a client, which is located in a remote branch, connected via 'WAN' to the 'LAN', where the 'opsi-configserver' and 'opsi-depotserver' are:

  • During communication with the 'opsi-configserver' small amounts of data are transferred, so there is no noticeable slowdown of the software deployment in a WAN.

  • But processing the 'Produktaktionen' might consume a very long time, depending on the packet sizes, bandwidth and latency of the 'WAN' connection. There also might occur timeouts during file access.

  • Therefore, during the installation is processing, the user has to wait for an unreasonably long time before logon is granted.

As an alternative to providing a dedicated 'opsi-depotserver' in the remote branch network, the remote clients can be connected via WAN/VPN extension module. Using the WAN/VPN extension module, the 'opsi-client-agent' can be configured this way:

  • At system startup, if there are no 'Produkte' cached and ready for installation, the user can logon immediately.

  • When there are 'Produktaktionen' set for the client, the 'opsiclientd', which is running on the client, starts downloading the required installation files from the 'opsi-depot' to the local file system. This is done in the background while the user is working. The maximum download bandwidth can be configured and also can be dynamically adapted to the current network traffic status.

  • When the synchronization of the 'Produkte' is completed, a reboot request is triggered.

  • The logged on user can accept the reboot request, or the client will be rebooted at some time later on.

  • At the next system startup, the cache is found to be filled with the 'Produkte' to be installed and the installation starts. In this case, the installation will use the downloaded files from the local file system, which increases the speed of installation even compared to a standard 'LAN' installation.

Now we examine the situation of a notebook, which at system startup often cannot connect the 'opsi-configserver':

  • Trying to connect the 'opsi-configserver' at system startup will result in a timeout.

  • Connecting the 'opsi-configserver' might be possible when a user logs on and a 'VPN' connection to the corporate network is established.

  • Without connection the 'opsi-configserver' no software deployment is available.

This situation also can be solved by using the WAN/VPN extension module.
The 'opsi-client-agent' can be configured the following way:

  • At system startup, if there are no 'Produkte' cached and ready for installation, the user can logon immediately.

  • Triggered by network activation or a by timer event, the 'opsiclientd' running on the client tries to connect the 'opsi-configserver'.

  • If the 'opsi-configserver' is reachable, the 'opsiclientd' starts to:

    • synchronize the configurations

    • download the required files from the 'opsi-depot' to the local file system.
      In combination with the opsi extension module 'Dynamic Depot Selection', the download is done from the best fitting 'opsi-depot'.

  • When the synchronization of the 'Produkte' is completed, a reboot request is triggered.

  • The logged on user can accept the reboot request or the client will be rebooted at some time later on.

  • At the next system startup, the cache is found to be filled with the 'Produkte' to be installed and the installation starts. In this case, the installation will use the downloaded files from the local file system, which increases the speed of installation even compared to a standard 'LAN' installation. So the 'opsiclientd' takes over the function of both the 'opsi-configserver' and the 'opsi-depotserver'.

  • At the next connect to the 'opsi-configserver' the results of the installation process (configuration change, log files …​) will be synchronized.

The download mechanism of product synchronization is multiple interruptible and will continue at the point of interruption. So files that are already downloaded will not have to be downloaded again.

The WAN/VPN extension module allows to connect clients, that are outside of the secure corporate network. Therefore additional security mechanisms are required regarding the communication between client and server.
So the 'opsiclientd' now offers the ability to verify the identity of an 'opsi-server'. Therefore the key pair of the SSL certificate of the 'opsiconfd' is used.
By this mechansim the 'opsi-configserver' as well as the 'opsi-depotserver' can be verified, assumed the communication is performed via 'opsiconfd' and 'SSL'. In case of an 'opsi-depot' the file access must be done by the 'opsiconfd' using 'HTTPS'/'WEBDAVS'. Access done via 'CIFS'/'SMB' will not be checked.

Caching of Produkte

Caching of 'Produkte' is done by the 'ProductCacheService', which is part of the 'opsiclientd'.
The 'ProductCacheService' synchronizes the local copy of an 'Produkt' with the current version of the corresponding 'Produkte' on the 'opsi-depot'. The location of the local product cache can be configured and defaults to %SystemDrive%\opsi.org\cache\depot.

Communication Protocol for accessing an opsi-depot

For transferring the product files, two different protocols are used:

  • 'CIFS'/'SMB'

  • 'HTTP(S)'/'WEBDAV(S)'

In case of using 'CIFS'/'SMB', a connection to the 'depotRemoteUrl' will be established as configured with the properties of the 'opsi-depot'. In case of using 'HTTP(S)'/'WEBDAV(S)', the 'depotWebdavUrl' is connected, which as well is to be configured with the properties of the 'opsi-depot'.

Which protocol is to be used, can be configured client specific by the 'Hostparameter' clientconfig.depot.protocol. Available values to be set as clientconfig.depot.protocol are cifs and webdav.

Also the 'opsi-linux-bootimage' is evaluating this setting and uses the specified protocol.
With opsiclientd, a different protocol can be used for individual events via the depot_protocol attribute.

Using the .files file for Synchronization

The synchronization process is based on the file <product-id>.files, which is located in the base directory of each 'Produkt' on the 'opsi-depot'. This file contains information about the files, directories ans symbolic links used by an 'Produkt'. Each line of that file contains such information. Different types of information are separated by a blank.
The first character of a line defines the type of the following entry. Available values are:

  • d for a directory

  • f for a file

  • l for a symbolic link

Separated by a blank follows the relative path, which is single quoted.
The next entry gives the sizes of the file (which is 0 for directories and symbolic links).
The final entry in case of a file is the MD5-sum of the file, in case of a symbolic link it is the target referred to by the symbolic link.

Example excerpt of a '.files' file:

d 'utils' 0
f 'utils/patch_config_file.py' 2506 d3007628addf6d9f688eb4c2e219dc18
l 'utils/link_to_patch_config_file.py' 0 '/utils/patch_config_file.py'

The '.files' file is generated automatically when installing 'Produkt-Pakete' (after running the postinst-Script).

When using the WAN/VPN extension, the files of 'Produkte' on the 'opsi-depot' should not be changed manually, otherwise the information contained in the '.files' file would be outdated, causing errors during the synchronization process.

Internal processing of Produkt caching

The synchronization of a local copy of an 'Produkt' processes as follows:

  • The '.files' file of the 'Produkt' is transferred to the local client.

  • Then it is checked, whether there is enough local disk space available to cache the 'Produkte'. If there isn’t enough disc space available, some old 'Produkte' will be deleted, which haven’t been used (synchronized) for a long time.

  • The local caching directory will be created if it doesn’t exist.

  • Referring to the '.files' file, any old files and directories, which aren’t in use anymore, will be deleted from the local 'Produkt' cache.

  • Then the '.files' file will be processed in the following order.

    • missing directories are created.

    • missing files are transferred.

    • existing files will be checked by size and MD5-sum and be synchronized again if necessary.

The synchronization results in an exact local copy of the 'Produkt' from the 'opsi-depot'.

On windows systems, no symbolic links will be created. Instead of links there will be copies of the link target.

When the 'Produkt' has completed successfully,

Configuring the Produkt caching

The 'Produkt' caching is configured in the section [cache_service] of the opsiclientd.conf.

  • product_cache_max_size (integer): The maximum size of the 'Produkt' cache in byte. This is important to limit the disk space to be used by 'Produkt' caching.

  • storage_dir (string): the path to the directory, in which the base directory depot for the 'Produkt' caching is to be created.

Further configurations can be done event specific.
Within an event configuration section [event_<event-config-id>] the following options are available:

  • cache_products (boolean): if the value of this option is true, in case of the event the 'ProductCacheService' will start to cache 'Produkte', for which a 'Produktaktion' is set. If additionally the value of the option use_cached_products is set to true, the further processing of this section will be suspended until the caching of 'Produkte' is completed.

  • cache_max_bandwidth (integer): the maximum bandwidth in byte/s to be used for caching. If this value is set to 0 or less, the bandwidth is unlimited.

  • cache_dynamic_bandwidth (boolean): if the value of this option is set to true, the bandwidth will be adapted dynamically. Therefore the network traffic at the network interface to the 'opsi-depot' will be monitored. If any traffic is detected, which is not caused by the 'Produkt' caching, the bandwidth for the caching will be sharply reduced, to allow other processes to work at (almost) full speed. If the caching works at reduced bandwidth and no more other network activity but the 'Produkt' caching is detected, the caching process will continue with unlimited bandwidth. The value of cache_max_bandwidth will be used to limit the maximum dynamic bandwidth.

  • use_cached_products (boolean): if the value of this option is set to true, the local 'Produkt' cache will be used for processing 'Produktaktionen'. If caching of the 'Produkte' is not completed, the processing of this event will stop and return an error code.

Caching of configurations

The caching of configurations is done by the 'ConfigCacheService', which is part of the 'opsiclientd'.
The 'ConfigCacheService' synchronizes a local 'Client-Cache-Backend' with the 'Config-Backend' of the 'opsi-configserver'.
The 'opsiclientd' provides per 'WebService' an access point to the backend and thereby provides quite the same functionality as the 'opsiconfd'.

The local 'Client-Cache-Backend'

The local 'Client-Cache-Backend' is based on 'SQLite' and mainly consists of a local working copy, a snapshot an a modification tracker, which records all changes of the local working copy.
The base directory of the config cache can be configured and defaults to %SystemDrive%\opsi.org\cache\config. The snapshot reflects the configuration status on the 'opsi-configserver' at the time of the last synchronization.
At the start of the processing, the local working copy is a copy of the snapshot, but will be modified during processing.

Internal processing of configuration synchronizing

The synchronization of the local changes of the 'Client-Cache-Backend' with the 'Config-Backend' of the 'opsi-configserver' is processed as follows:

  • The changes of the local working copy registered by the modification-tracker are transferred to the 'opsi-configserver'. Any changes of the configuration on the 'opsi-configserver' since the last synchronization will be detected by comparing to the snapshot. If there are any conflicts deteced, the following rules apply:

    • In case of inventory data, the local client data have priority.

    • For 'Action-Requests' the value of the client is valid, unless the version of the corresponding package has changed in the meantime on the server side. Then the server value is preferred.

    • In case of 'Installationsstatus' and 'Aktionsergebnis', the client data have priority.

    • If the opsi licenense management modul is switched on (config: 'license-management.use=true'), the config server tries to find a license pool for the product by the assignment pool to productId. I free license of this pool will be reserved and this license will be replicated. Any unused licences, which have been reserved during replication, will be released again.

    • The new state of 'Hostparametern' and 'Product-Properties' is only transferred to the server if they have not been changed server-side in the meantime.

  • The modification tracker will be cleared.

  • The logfiles will be transferred.

The 'Config-Backend' replication of the 'opsi-configserver' to the 'Client-Cache-Backend' is processed as follows:

  • The replication only takes place, if any 'Action-Requests' are set on the 'opsi-configserver'. The 'Produktaktion' always does not count in this respect. The replication process will start only if the status of 'Action-Requests' is changed since the last replication.

  • The modification tracker and the local working copy are cleared.

  • The configurations required for local processing will be replicated.

  • If 'Action-Requests' are set for 'Produkte' which are marked as to require a license, the required software license will be reserved from a 'Lizenzpool', which is assigned to that 'Produkt'.

  • Additionally required data, as there are the auditHardwareConfig and the modules, will be transferred.

  • The snapshot and the local working copy will be updated, so they have the same content.

A successful replication from server to client results in:

The sync completed is running until the next reboot, or until it is canceled by manually fired event (e.g. on_demand). In the latter case the config cache is marked invalid (which implies that the config has to be resynced - in case of changes) and the other event is processed.

Configuration of config caching

The configuration of the config caching is mainly done event specific:
Within an event configuration section [event_<event-config-id>], the following options are available:

  • sync_config_to_server (boolean): if the value of this option is set to true, the 'ConfigCacheService' in case of that event starts to transfer the changes registered by the modification tracker to the 'opsi-configserver'. The process will wait for that task to complete.

  • sync_config_from_server (boolean): if this value is set to true, the 'ConfigCacheService' starts with the replication. If additionally the value of the option use_cached_config is set to true, the processing of this event is suspended until the replication is completed.

  • use_cached_config (boolean): if the value of this option is set to true, the 'Client-Cache-Backend' will be used for processing the 'Produktaktionen'. If the synchronization is not completed, the processing of this event will be stopped and return an error code.

  • post_sync_config_to_server (boolean): has the same functionality as sync_config_to_server, but will be evaluated after the 'Produktaktionen' have been completed.

  • post_sync_config_from_server (boolean): has the same functionality as sync_config_from_server, but will be evaluated after the 'Produktaktionen' have been completed.

The 'opsi-client-agent'-package comes with a opsiclientd.conf prepared for the WAN/VPN extension.
For activating the WAN/VPN extension, just enabling of some events and disabling of some others is required.
The configuration of the 'opsi-client-agent' also can be done from the web service (see: opsi-client-agent web service), so it is recommended to create the following 'Hostparameter':

  • opsiclientd.event_gui_startup.active (boolean, default: true)

  • opsiclientd.event_gui_startup\{user_logged_in\}.active (boolean, default: true)

  • opsiclientd.event_net_connection.active (boolean, default: false)

  • opsiclientd.event_timer.active (boolean, default: false)

By these 'Hostparameter', events can be enabled or disabled client specific. The 'Hostparameter' can be created using the 'opsi-configed' or 'opsi-admin'.

For creating the 'Hostparameter' by 'opsi-admin', the following commands have to be executed on the 'opsi-configserver':

opsi-admin -d method config_createBool opsiclientd.event_gui_startup.active "gui_startup active" true
opsi-admin -d method config_createBool opsiclientd.event_gui_startup{user_logged_in}.active "gui_startup{user_logged_in} active" true
opsi-admin -d method config_createBool opsiclientd.event_net_connection.active "event_net_connection active" false
opsi-admin -d method config_createBool opsiclientd.event_timer.active "event_timer active" false

The default values are as they come with the special opsiclientd.conf.

If you do not set the defaults like described above and skip directly to the commands below you set all your clients into WAN mode !

For a WAN/VPN client, which shall do caching of configurations and 'Produkte', the 'Hostparameter' have to be configured as follows:

  • opsiclientd.event_gui_startup.active: false

  • opsiclientd.event_gui_startup\{user_logged_in\}.active: false

  • opsiclientd.event_net_connection.active: true

  • opsiclientd.event_timer.active: true

The client specific 'Hostparameter' can be set by 'opsi-configed' or 'opsi-admin'.

To set the 'Hostparameter' by 'opsi-admin', the following commands have to be executed on the 'opsi-configserver': (in this example the client has the 'opsi-host-Id' vpnclient.domain.de):

opsi-admin -d method configState_create opsiclientd.event_gui_startup.active vpnclient.domain.de false
opsi-admin -d method configState_create opsiclientd.event_gui_startup{user_logged_in}.active vpnclient.domain.de false
opsi-admin -d method configState_create opsiclientd.event_net_connection.active vpnclient.domain.de true
opsi-admin -d method configState_create opsiclientd.event_timer.active vpnclient.domain.de true

This configuration will process as follows:

  • At system start of the client there will be no connection established to the 'opsi-configserver'.

  • When the activation of a network interface is detected, a connection to the 'opsi-configserver' will be established (if possible) and the synchronization starts as background task.

  • A timer-Event will be established, which tries at regular intervals to trigger the synchonization process.

The caching of 'Produkte' can be done via the protocols 'HTTPS'/'WEBDAVS' or 'CIFS'/'SMB'.

When using 'webdav', access to the 'opsi-depot' is performed by the 'opsiconfd'.

  • advantages:

    • easy firewall configuration, for it requires just port 4447.

    • verify of the SSL-certificate of the 'opsi-depot' available.

  • disadvantage:

    • the 'opsiconfd' generates more traffic on the 'opsi-depot'.

By using 'cifs', access to the 'opsi-depot' is done via 'SAMBA'.

  • advantage:

    • the 'SAMBA'-server shows a good performance, is resource-conserving and well scaleable.

  • disadvantages:

    • the firewall configuration is more complicated, access to the SAMBA ports is required.

    • no verify of the SSL-certificate of the 'opsi-depot' is available.

An instruction for configuring the protocols is to be found in the chapter Communication Protocol for accessing an opsi-depot.

ospclientd-infopage-wan-cached
Figure 1. processing of an installation with WAN extension as displayed in the opsiclientd infopage

To activate the verifying of SSL certificates, in the opsiclientd.conf within the section [global], the option verify_server_cert is to be set to true. This, during connection to an opsiconfd, results in verifying the 'opsi-server' by using the SSL certificate. The server certificates will be stored in the directory %SystemDrive%\opsi.org\opsiclientd\server-certs. The name of the certificate is combined from the server address (IP or name) and the extension .pem. If at connection time no stored certificate is to be found, no checking will be done.

To publish a changed certificate, the old certificate stored on the clients has to be deleted. This can be done by the RPC-method deleteServerCerts, which is available from the control interface of the 'opsiclientd'.