All the documentation and resources related to the Tigase server monitoring.
Tigase server can be remotely monitored over following protocols: JMX/RMI
, SNMP
and HTTP
. Even though JMX
offers the biggest control and visibility to the server states, all of the monitoring services give the same basic set of the server statistics:
JMX/RMI and SNMP servers offer basic security and can restrict access while the HTTP server doesn’t offer any access restriction mechanisms. Therefore HTTP monitoring is recommended to operate behind a firewall.
The monitoring itself causes very low overhead in terms of the resources and CPU consumption on top of the normal Tigase processing requirements so it can be left on without worrying about performance degradation.
NOTE This works with the Tigase server from version 4.2.0 or build 1418.
Statistics binaries are built-in -dist-max
and no extra files are needed. If you have downloaded -dist
file, you will need tigase-extras[https://tigase.tech/projects/tigase-extras/repository] built and included in the jars/
directory.
You can either run the Tigase installer and use the configuration wizard to activate the monitoring or edit etc/config.tdsl file and add following lines:
monitoring() { jmx() { port = 9050 } http() { port = 9080 } snmp() { port = 9060 } }
As you see there is a separate block for each monitoring server you want to activate. Each server is responsible for activation of a different protocol and takes a single parameter - port number. There are following protocols supported right now:
jmx
- activating monitoring via JMX/RMIhttp
- activating monitoring over HTTP protocolsnmp
- activating monitoring over SNMP protocolYou can have all protocols active at the same time or any combination of them or none.
Both JMX and SNMP offer security protection to limit access to monitoring data. The security configuration is a bit different for both.
After the server installation or in the SVN repository you can find 2 files in the etc/
directory: jmx.access
and jmx.password
.
jmx.access
is a user permission file. You can use it to specify whether the user can access the monitoring data for reading only readonly
or with read-write readwrite
access. There are example entries in the file already and the content may simply look like:
monitor readonly admin readwrite
jmx.password
is a user password file. You can set user passwords here and the format again is very simple and the same as for jmx.access. There are example entries already provided for you convenience. Content of the file may look like the example below:
admin admin_pass monitor monitor_pass
Using above to files you can control who and how can access the JMX monitoring services.
Access to SNMP monitoring is controlled using ACL (access control lists) which can be configured in the file snmp.acl
located in etc/
directory. It contains lots of detailed instructions how to setup ACL and restrict access per user, host and what kind access is allowed. The simplest possible configuration may look like this:
acl = { { communities = public, private access = read-only managers = public.host.com, private.host.com } { communities = admin access = read-write managers = localhost, admin.host.com } }
You might also need Tigase MIB definition: TIGASE-MANAGEMENT-MIB.mib for the server specific statistics. The MIB contains definition for all the server statistics exposed via SNMP.
By default we can retrieve server statistics using XMPP, no additional setup is necessary.
Accessing statistics over XMPP protocol requires any XMPP client capable of executing XEP-0050: Ad-Hoc Commands. It’s essential to remember, that only administrator (a user whose JID is configured as administrative) can access the statistics.
For the purpose of this guide Psi client will be used. After successfully configuring and connecting to account with administrative privileges we need to access Service Discovery, either from application menu or from context menu of the particular account account:
In the Service Discovery window we need to find Server Statistics component:
We can either access statistics for all components or select particular component after expanding the tree. To execute ad-hoc command simply double click on the particular node which will open window with statistics:
In this window, in addition to see the statistics, we can adjust Stats level by selecting desired level from the list and confirm by clicking Finish.
In order to access statistics over JMX we need to enable support for it in Tigase - Monitoring Activation. Afterwards we can use a number of tools to get to the statistics, for example the following:
After opening JConsole we either select local process or provide details of the remote process, including IP, port and credentials from etc/jmx.* files:
Afterwards we navigate to the MBeans tab from where we can access the tigase.stats
MBean. It offers similar options to XMPP - either accessing statistics for all components or only for particular component as well as adjusting level for which we want to obtain statistics:
In order to collect statistics over period of time following groovy script can be used: StatsDumper.groovy. It’s a Simple JMX client that connects to Tigase and periodically saves all statistics to files.
It takes following parameters:
$ groovy StatsDumper.groovy [hostname] [username] [password] [dir] [port] [delay(ms)] [interval(ms)] [loadhistory(bool)]
hostname
- address of the instanceusername
- JMX usernamepassword
- JMX usernamedir
- directory to which save the files with statisticsport
- port on which to make the connectiondelay
(ms) - initial delay in milliseconds after which statistics should be savedinterval
(ms) - interval between each retrieval/saving of statisticsloadhistory
(bool) - indicates whether or not load statistics history from server (if such is enabled in Tigase)Tigase includes an eventbus component to help with monitoring has been implemented. This allows you to set thresholds for certain predefined tasks and you or other JIDs can be sent a message when those thresholds are passed. You can even configure a mailer extension to have an E-mail sent to system administrators to let them know an event has occurred! Lets begin with setup and requirements.
Eventbus is based on a limited PubSub specification. Events are delivered to subscribers as a normal PubSub notification.
Each component or client may subscribe for specific types of events. Only components on cluster nodes are allowed to publish events.
Events in Eventbus are identified by two elements: name of event and its namespace:
<EventName xmlns="tigase:demo"> <sample_value>1</sample_value> </EventName>
Where event name is EventName
and namespace is tigase:demo
.
Listeners may subscribe for a specific event or for all events with specific a namespace. Because in pubsub, only one node name exists, so we have to add a way to convert the event name and namespace to a node name:
nodename = eventname + "|" + namespace
So for example, to subscribe to <EventName xmlns="tigase:demo">
, node must be: EventName|tigase:demo
. If you wish to subscribe to all events with a specific namespace, use an asterisk (*
) instead of the event name: *|tigase:demo
.
If client is subscribed to *|tigase:demo node
, then events will not be sent from node *|tigase:demo
, but from the real node (in this case: EventName|tigase:demo
).
Eventbus monitoring has several pre-defined tasks that can be monitored and set to trigger. What follows is the list of tasks with the options attributed to each task.
disk-task - Used to check disk usage. Available Options
enabled
- Enable or disable task, Boolean value.period
- Period of running check, Integer value.threshold
- Percentage of used space on disk, Float value.cpu-temp-task - Used to check CPU temperature. Available Options
enabled
- Enable or disable task, Boolean value.period
- Period of running check, Integer value.cpuTempThreshold
- Temperature threshold of CPU in °C.load-checker-task - Used to check system load. Available Options
enabled
- Enable or disable task, Boolean value.period
- Period of running check, Integer value.averageLoadThreshold
- Average percent load threshold, Long value.memory-checker-task - Used to check memory usage. Available Options
enabled
- Enable or disable task, Boolean value.period
- Period of running check, Integer value.maxHeapMemUsagePercentThreshold
- Alarm when percent of used Heap memory is larger than, Integer value.maxNonHeapMemUsagePercentThreshold
- Alarm when percent of used Non Heap memory is larger than, Integer value.logger-task - Used to transmit log entries depending on level entered.
enabled
- Enable or disable task, Boolean value.levelThreshold
- Minimal log level that will be the threshold. Possible values are SEVERE, WARNING, INFO, CONFIG, FINE, FINER, FINEST, and ALL.connections-task - Used to check users disconnections. NOTE: The event will be generated only if both thresholds (amount and percentage) will be fulfilled.
enabled
- Enable or disable task, Boolean value.period
- Period of running check in ms, Integer value.thresholdMinimal
- Minimal amount of disconnected users required to generate alarm.threshold
- Minimal percent of disconnected users required to generate alarm.Configuration of the eventbus monitor can be done one of two ways; either by lines in config.tdsl
file, or by sending XMPP stanzas to the server. You may also send XMPP stanzas VIA HTTP REST.
XMPP stanza configurations will override ones in config.tdsl, but they will only last until the server restarts.
Tasks can be configured in the config.tdsl
file. See available tasks for the tasks that can be setup.
To enable a specific monitor task, use the following line:
monitor { '$TASKNAME' { setting = value } }
Where monitor is the component name for MonitorComponent
, and $TASKNAME
is one of the available task names.
This format will be the same for other settings for tasks, and it’s best to group settings under one heading. For example:
monitor { 'connections-task' { enabled = true period = 1000 } }
sets the check period to 1000 milliseconds and enables connections-task
.
Once triggers have been activated, they will become dormant. Think of these as one-shot settings.
We can also configure the eventbus monitor component using XMPP stanzas. This allows us to set and change configurations during server runtime. This is done using a series of iq
stanzas send to the monitor component.
We can query each component for its current settings using the following stanza.
<iq type="set" to="monitor@$DOMAIN/disk-task" id="aad0a"> <command xmlns="http://jabber.org/protocol/commands" node="x-config"/> </iq>
The server will return the component current settings which will make things easier if you wish to edit them. In this case, the server has returned the following to us
<iq from="monitor@$DOMAIN/disk-task" type="result" id="aad0a" to="alice@coffeebean.local/Psi+"> <command xmlns="http://jabber.org/protocol/commands" status="executing" node="x-config" sessionid="0dad3436-a029-4082-b0e0-04d838c6c0da"> <x xmlns="jabber:x:data" type=""> <title>Task Configuration</title> <instructions/> <field type="boolean" label="Enabled" var="x-task#enabled"> <value>0</value> </field> <field type="text-single" label="Period [ms]" var="x-task#period"> <value>60000</value> </field> <field type="text-single" label="Disk usage ratio threshold" var="threshold"> <value>0.8</value> </field> </x> </command> </iq>
This tells us that the disk-task setting is not active, has a period of 60000ms, and will trigger when disk usage is over 80%.
To send new settings to the monitor component, we can send a similar stanza back to the monitor component.
<iq type="set" to="monitor@$DOMAIN/disk-task" id="aad1a"> <command xmlns="http://jabber.org/protocol/commands" node="x-config" sessionid="0dad3436-a029-4082-b0e0-04d838c6c0da"> <x xmlns="jabber:x:data" type="submit"> <field type="boolean" var="x-task#enabled"> <value>0</value> </field> <field type="text-single" var="x-task#period"> <value>60000</value> </field> <field type="text-single" var="threshold"> <value>0.8</value> </field> </x> </command> </iq>
To which a successful update will give you an XMPP success stanza to let you know everything is set correctly.
Alternatively, you can update specific settings by editing a single field without adding anything else. For example, if we just wanted to turn the disk-task on we could send the following stanza:
<iq type="set" to="monitor@$HOSTNAME/disk-task" id="ab53a"> <command xmlns="http://jabber.org/protocol/commands" node="x-config"> <x xmlns="jabber:x:data" type="submit"> <field type="boolean" var="x-task#enabled"> <value>1</value> </field> </x> </command> </iq>
To set any other values, do not forget that certain parts may need to be changed, specifically the
<field type="boolean" var=x-task#enabled">
fields:
var=x task#
will be followed by the property value taken directly from the Available Tasks section.Without a place to send messages to, eventbus will just trigger and shut down. There are two different methods that eventbus can deliver alarm messages and relevant data; XMPP messages and using the mailer extension.
In order to retrieve notifications, a subscription to the eventbus@tigase.org
user must be made.
Keep in mind that subscriptions are not persistent across server restarts, or triggers.
The eventbus schema is very similar to most XMPP subscription requests but with a few tweaks to differentiate it if you wanted to subscribe to a certain task or all of them. Each task is considered a node, and each node has the following pattern: eventName|eventXMLNS
. Since each monitoring task has the tigase:monitor:event
event XMLNS, we just need to pick the event name from the list of tasks.
So like the above example, our event node for the disk task will be disk-task|tigase:monitor:event
.
Applied to an XMPP stanza, it will look something like this:
<iq type='set' to='eventbus@tigase.org' id='sub1'> <pubsub xmlns='http://jabber.org/protocol/pubsub'> <subscribe node='disk-taskEvent|tigase:monitor:event' jid='$USER_JID'/> </pubsub> </iq>
Don’t forget to replace $USER_JID
with the bare JID of the user you want to receive those messages. You can even have them sent to a MUC or any component with a JID.
Available events are as follows:
disk-task
logger-task
memory-checker-task
load-checker-task
cpu-temp-task
connections-task
Alternatively, you can also subscribe to all events within the eventbus by using a wildcard * in place of the event XMLNS like this example:
<iq type='set' to='eventbus@tigase.org' id='sub1'> <pubsub xmlns='http://jabber.org/protocol/pubsub'> <subscribe node='*|tigase:monitor:event' jid='$USER_JID'/> </pubsub> </iq>
<message from='eventbus.shakespeare.lit' to='francisco@denmark.lit' id='foo'> <event xmlns='http://jabber.org/protocol/pubsub#event'> <items node='EventName|tigase:demo'> <item> <EventName xmlns="tigase:demo" eventSource="samplecomponent.shakespeare.lit'" eventTimestamp="1444216850"> <sample_value>1</sample_value> </EventName> </item> </items> </event> </message>
Tigase Server Monitor Mailer Extension (TSMME) can send messages from the monitor component to a specified E-mail address so system administrators who are not logged into the XMPP server.
For v7.1.0 versions and later, TSMME is already included in your distribution package and no extra installation is needed.
For versions older than 7.1.0 TSMME requires two files to operate:
.jar
file into jars/
directory.javax.mail.jar
file which may be downloaded from this link. Also place this file in the jars/
directory.Tigase Mailer Extension may be configured via the config.tdsl
file in the following manner:
monitor { 'mailer-from-address' = 'sender@tigase.org' 'mailer-smtp-host' = 'mail.tigase.org' 'mailer-smtp-password' = '********' 'mailer-smtp-port' = '587' 'mailer-smtp-username' = 'sender' 'mailer-to-addresses' = 'receiver@tigase.org,admin@tigase.org' }
Here is an explanation of those variables.
mailer-smtp-host
- SMTP Server hostname.mailer-smtp-port
- SMTP Server port.mailer-smtp-usernam
- name of sender account.mailer-smtp-password
- password of sender account.mailer-from-address
- sender email address. It will be set in field from in email.mailer-to-addresses
- comma separated notification receivers email addresses.It is recommended to create a specific e-mail address in your mail server for this purpose only, as the account settings are stored in plaintext without encryption.
It is possible to enable and configure automatic storage of statistics information. To do that you need to configure any of following statistics loggers
as a StatisticsCollector
component sub-beans:
tigase.stats.CounterDataArchivizer
tigase.stats.CounterDataLogger
tigase.stats.CounterDataFileLogger
As an example to configure tigase.stats.CounterDataFileLogger
to archive statistics data with level FINE
every 60 seconds to file prefixed with stat
and located in logs/server_statistics
following entry is needed:
stats() { 'stats-file-logger' (class: tigase.stats.CounterDataFileLogger) { 'stats-directory' = 'logs/server_statistics' 'stats-filename' = 'stat' 'stats-unixtime' = false 'stats-datetime' = true 'stats-datetime-format' = 'HH:mm:ss' 'stats-level' = 'FINEST' } }