Welcome to Tigase Unified Archive component guide
1. Overview
Tigase Unified Archive Component is extended version of Tigase Message Archiving Component allow you to store not only messages but also information about:
-
login and logout events
-
jingle call events
-
offline messages
It is based on extended implementation of XEP-0136: Message Archiving with few custom features allowing more advance searches.
2. Announcement
2.1. Major changes
Tigase Unified Archive component has undergone a few major changes to our code and structure. To continue to use Tigase Unified Archive component, a few changes may be needed to be made to your systems. Please see them below:
2.1.1. Database schema changes
We decided to no longer use in-code database upgrade to update database schema of Unified Archive component and rather provide separate schema files for every supported database.
Additionally we moved from in-code generated SQL statements to stored procedures which are now part of provided database schema files.
To continue usage of new versions of Unified Archive component it is required to manually load new component database schema, see Preparation of database section for informations about that.
Warning
|
Loading of new database schema is required to use new version of Unified Archive component. |
2.2. New features
2.2.1. Support for using separate database for different domains
Since this version it is possible to use separate archived messages based on domains. This allows you to configure component to store archived message for particular domain to different database.
For more informations please look into [Using separate store for archived messages]
3. Features
Tigase Unified Archive Component contains few additional features useful for working with archives containing not only messages but also other informations.
3.1. Archive of Jingle call events
For instructions on how to enable Jingle call archive see Enabling Jingle call archive section.
3.1.1. Retrieving a list of collections
To retrieve collections with informations about Jingle call events item-type
attribute needs to be added to <list/>
element from section 7.1 Retrieving a List of Collections of XEP-0136 specification with value set to call
.
A result it will be returned as specified in XEP-0136, but <list/>
element will contain <events/>
element for each collections instead of <chat/>
element.
Additionally <list/>
element will have item-type
attribute set to call
.
Example
Request to retrieve collection of calls with juliet@capulet.com
between 2014-01-01 00:00:00
and 2014-05-01 00:00:00
<iq type="get" id="query1">
<list xmlns='urn:xmpp:archive'
with='juliet@capulet.com'
start='2014-01-01T00:00:00Z'
end='2014-05-01T00:00:00Z'
item-type='call'/>
</iq>
Response with single collection starting at 2014-02-01 13:01:34
<iq type="result" id="query2">
<list xmlns='urn:xmpp:archive' item-type='call'>
<events with='juliet@capulet.com' start='2014-02-01T13:01:34Z'/>
<set xmlns="http://jabber.org/protocol/rsm">
<first index='0'>0</first>
<last>0</last>
<count>1</count>
</set>
</events>
</iq>
3.1.2. Retrieve a list of entries from collection
To retrieve items with informations about Jingle call from collection item-type
attribute needs to be added to <retrieve/>
element from section 7.2 Retrieving a Collection of XEP-0136 specification with value set to call
.
A result will be returned as specified in XEP-0136, but instead of <chat/>
element <events/>
element will be returned.
Moreover instead of message subelements in <from/>
and <to/>
elements, there will be custom element containing information about call, ie. Jingle session ID - <sid/>
.
Example
Request to retrieve calls events with juliet@capulet.com
between 2014-02-01 13:00:00
and 2014-05-01 00:00:00
<iq type="get" id="query1">
<retrieve xmlns='urn:xmpp:archive'
with='juliet@capulet.com'
start='2014-02-01T13:00:00Z'
end='2014-05-01T00:00:00Z'
item-type='call'/>
</iq>
Response with single entry of incoming call at 2014-02-01 13:01:34 with call SID.
<iq type="result" id="query2">
<events xmlns='urn:xmpp:archive' with='juliet@capulet.com' start='2014-02-01T13:00:00Z' item-type='call'>
<from secs='94' item-type='call'><sid>SID_OF_CALL</sid></from>
<set xmlns="http://jabber.org/protocol/rsm">
<first index='0'>0</first>
<last>0</last>
<count>1</count>
</set>
</events>
</iq>
3.2. Archive of login/logout events
For instructions on how to enable login history please see Enabling login history section.
3.2.1. Retrieving a list of collections
To retrieve collections with informations about user logins item-type
attribute needs to be added to <list/>
element from section 7.1 Retrieving a List of Collections of XEP-0136 specification with value set to login
.
A result it will be returned as specified in XEP-0136, but <list/>
element will contain <events/>
element for each collections instead of <chat/>
element.
Additionally <list/>
element will have item-type
attribute set to login
.
Example
Request to retrieve collection of login events of juliet@capulet.com
between 2014-01-01 00:00:00
and 2014-05-01 00:00:00
(sent from juliet@capilet.com
account)
<iq type="get" id="query1">
<list xmlns='urn:xmpp:archive'
with='juliet@capulet.com'
start='2014-01-01T00:00:00Z'
end='2014-05-01T00:00:00Z'
item-type='login'/>
</iq>
Response with single collection starting at 2014-02-01 13:01:34
<iq type="result" id="query2">
<list xmlns='urn:xmpp:archive' item-type='login'>
<events with='juliet@capulet.com' start='2014-02-01T13:01:34Z'/>
<set xmlns="http://jabber.org/protocol/rsm">
<first index='0'>0</first>
<last>0</last>
<count>1</count>
</set>
</events>
</iq>
3.2.2. Retrieve a list of entries from collection
To retrieve items with informations about user logins from collection item-type
attribute needs to be added to <retrieve/>
element from section 7.2 Retrieving a Collection of XEP-0136 specification with value set to login
.
A result will be returned as specified in XEP-0136, but instead of <chat/>
element <events/>
element will be returned.
Moreover instead of message subelements in <from/>
element, there will be custom element containing information about session, ie. <bind/>
element with connection resource set as a value. There will be also type
attribute added to <from/>
element with value set to login
to mark that this is login
event.
Example
Request to retrieve login events of juliet@capulet.com
between 2014-02-01 13:00:00
and 2014-05-01 00:00:00
(sent from juliet@capilet.com
account)
<iq type="get" id="query1">
<retrieve xmlns='urn:xmpp:archive'
with='juliet@capulet.com'
start='2014-02-01T13:00:00Z'
end='2014-05-01T00:00:00Z'
item-type='login'/>
</iq>
Response with single login entry at 2014-02-01 13:01:34 with call SID.
<iq type="result" id="query2">
<events xmlns='urn:xmpp:archive' with='juliet@capulet.com' start='2014-02-01T13:00:00Z' item-type='login'>
<from secs='94' item-type='login'><bind>balcony</bind></from>
<set xmlns="http://jabber.org/protocol/rsm">
<first index='0'>0</first>
<last>0</last>
<count>1</count>
</set>
</events>
</iq>
3.3. Retrieval of all events and messages
It is also possible to retrieve full history of login/logout events, and history of Jingle calls and messages.
3.3.1. Retrieve a list of entries
To retrieve items with informations about every archived entry item-type
attribute needs to be added to <retrieve/>
element from section 7.2 Retrieving a Collection of XEP-0136 specification with value set to any
.
A result will be returned as specified in XEP-0136, but instead of <chat/>
element <events/>
element will be returned.
<from/>
and <to/>
elements will have item-type
attribute set to message
, login
, logout
or call
depending on type of returned event. Subelements of <from/>
and <to/>
elements will depend on type of archived event it represents.
Example
Request to retrieve events of juliet@capulet.com
between 2014-02-01 13:00:00
and 2014-05-01 00:00:00
(sent from juliet@capilet.com
account)
<iq type="get" id="query1">
<retrieve xmlns='urn:xmpp:archive'
with='juliet@capulet.com'
start='2014-02-01T13:00:00Z'
end='2014-05-01T00:00:00Z'
item-type='any'/>
</iq>
Response with single login entry at 2014-02-01 13:01:34 with call SID.
<iq type="result" id="query2">
<events xmlns='urn:xmpp:archive' with='juliet@capulet.com' start='2014-02-01T13:00:00Z'>
<from secs='34' item-type='message'><body>Example message</body></from>
<from secs='64' item-type='call'><sid>SID_OF_CALL</sid></from>
<from secs='94' item-type='login'><bind>balcony</bind></from>
<set xmlns="http://jabber.org/protocol/rsm">
<first index='0'>0</first>
<last>2</last>
<count>3</count>
</set>
</events>
</iq>
3.4. Retrieval of recent events
It is now possible to retrieve list of last logged events (chat, calls, groupchats) for every bare jid with which archive owner communicated with. As this collection may be rather big, we added support for XEP-0059: Result Set Management to make it possible to request part of a list (ie. first 100 entries). For now it is only possible to request next entries of a list by providing offset/index at which next result should start at.
Warning
|
There is no support for RSM pagination done using <before/> and <after/> .
|
3.4.1. Retrieval of list of all recent events
In this example we will request and receive recent events of any type or condition from whole time for which we have entries in an archive.
<iq type="get" id="query1">
<query xmlns="urn:xmpp:tigase:recent"/>
</iq>
<iq id="1" xmlns="jabber:client" type="result">
<events xmlns="urn:xmpp:tigase:recent">
<event
with="buddy-2@example.com"
type="call"
direction="from"
condition="success"
time="2017-01-21T10:06:10Z" />
<event
with="some-room@muc.example.com"
type="groupchat"
direction="from"
body="Welcome to our groupchat"
time="2017-01-20T10:08:12Z"/>
<event
with="buddy-1@example.com"
type="chat"
direction="to"
body="Hello world"
time="2017-01-19T17:20:51Z" />
</events>
</iq>
3.4.2. Retrieval of list of all recent events in specified time period
In this example we will request and receive recent events of any type or condition which occurred within specified time frame.
<iq type="get" id="query1">
<query xmlns="urn:xmpp:tigase:recent" start="2017-01-20T00:00:00Z" end="2017-01-31T00:00:00Z"/>
</iq>
<iq id="1" xmlns="jabber:client" type="result">
<events xmlns="urn:xmpp:tigase:recent">
<event
with="buddy-2@example.com"
type="call"
direction="from"
condition="success"
time="2017-01-21T10:06:10Z" />
<event
with="some-room@muc.example.com"
type="groupchat"
direction="from"
body="Welcome to our groupchat"
time="2017-01-20T10:08:12Z"/>
</events>
</iq>
3.4.3. Requesting filtered list of recent events
It is also possible to filter list of recent events by event type
-
chat
-
groupchat
-
call (and call status)
-
success (successful call)
-
missed (missed call)
-
canceled (canceled call)
-
Requesting only recent chat events
<iq type="get" id="query1">
<query xmlns="urn:xmpp:tigase:recent">
<event-types>
<chat/>
</event-types>
</query>
</iq>
<iq id="1" xmlns="jabber:client" type="result">
<events xmlns="urn:xmpp:tigase:recent">
<event
with="buddy-2@example.com"
type="chat"
direction="from"
condition="success"
time="2017-01-21T09:06:10Z" />
<event
with="buddy-1@example.com"
type="chat"
direction="to"
body="Hello world"
time="2017-01-19T17:20:51Z" />
</events>
</iq>
Requesting only recent missed or canceled call events
<iq type="get" id="query1">
<query xmlns="urn:xmpp:tigase:recent">
<event-types>
<call>
<missed/>
<canceled/>
</call>
</event-types>
</query>
</iq>
<iq id="1" xmlns="jabber:client" type="result">
<events xmlns="urn:xmpp:tigase:recent">
<event
with="buddy-2@example.com"
type="call"
direction="from"
condition="missed"
time="2017-01-20T10:06:10Z" />
<event
with="buddy-1@example.com"
type="call"
direction="to"
condition="canceled"
time="2017-01-18T17:20:51Z" />
</events>
</iq>
4. Database
4.1. Preparation of database
Before you will be able to use Tigase Unified Archive Component and store messages and events in particular database you need to initialize this database. We provide few schemas for this component for MySQL, PostgreSQL, SQLServer and DerbyDB.
They are placed in database/
directory of installation package and named in dbtype-unified-archive-version.sql
, where dbname
in name of database type which this schema supports and version
is version of a Unified Archive Component for which this schema is designed.
You need to manually select schema for correct database and component and load this schema to database. For more information about loading database schema look into [Database Preparation] section of [Tigase XMPP Server Administration Guide]
4.2. Upgrade of database schema
Database schema for our components may change between versions and if so it needs to be updated before new version may be started. To upgrade schema please follow instuctions from Preparation of database section.
Note
|
If you use SNAPSHOT builds then schema may change for same version as this are versions we are still working on. |
4.3. Schema description
Tigase Unified Archive component uses few tables and stored procedures and it shares same tables with Tigase Message Archiving component - in fact it uses same tables with addition of few fields.
Due to that used tables are prefixed with tig_ma_
, but used stored procedures are prefixed with Tig_UA_
.
4.3.1. Table tig_ma_jids
This table stores all jids related to stored messages and events, ie. from to
and from
attributes of archived stanzas.
Field | Description | Comments |
---|---|---|
jid_id |
Database ID of a JID |
|
jid |
Value of a bare JID |
|
jid_sha1 |
SHA1 value of lowercased bare JID |
Used for proper bare JID comparison during lookup. (Not exists in PostgreSQL schema) |
domain |
Domain part of a bare JID |
Stored for easier lookup of messages owned by users of a particular domain |
4.3.2. Table tig_ma_msgs
Table stores archived events.
Field | Description | Comments |
---|---|---|
msg_id |
Database ID of a event |
|
owner_id |
ID of a bare JID of a event owner |
References |
buddy_id |
ID of a bare JID of a event recipient/sender (different than owner) |
References |
buddy_res |
Resource part of a event recipient/sender JID |
|
ts |
Timestamp of a event |
Timestamp of archivization or delayed delivery |
direction |
Direction of event |
0 - sent by owner 1 - recevied by owner |
type |
Stanza type |
Value of message |
body |
Body of a message |
|
msg |
Serialized event |
|
stanza_hash |
Hash of event parts |
It is used to make sure that event is stored only once |
item-type |
Event type |
May be one of: |
offline |
Marks offline events |
It is set to 1 when it stores offline event, not yet delivered to client |
4.3.3. Table tig_ma_tags
Table stores tags of archived messages. It stores one tag for many messages using tig_ma_msgs_tags
to store relation between tag and a message.
Field | Description | Comments |
---|---|---|
tag_id |
Database ID of a tag |
|
owner_id |
ID of a bare JID of a tag owner |
ID of bare JID of owner for which messages with this tag were archived |
tag |
Actual tag value |
5. Configuration
To enable Tigase Unified Archive Component you need to add following block to etc/init.properties
file:
unified-archive () { }
It will enable component and configure it under name unified-archive
.
By default it will also use database configured as default
data source to store data.
Due to fact that Tigase Unified Archive component is extended version of Tigase Message Archiving component it shares configuration properties.
Every configuration property of message-archive
component may be set to unified-archive
component. In same way every configuration property of message-archive
processor may be set to unified-archive
processor.
5.1. Support for XEP-0136
To be able to use Unified Archive component with XEP-0136: Message Archiving protocol, you additionally need to enable unified-message-archive-xep-0136
SessionManager processor:
sess-man { unified-message-archive-xep-0136 () { } }
5.2. Enabling Jingle call archive
To enable archiving of Jingle calls you need to enable jingle-archive
SessionManager processor:
sess-man { jingle-archive () { } }
Additionally you may want to replace urn:xmpp:jingle:1
XMLNS in entries stored in Unified Archive. For that you need to set omitJingleXMLNS
of jingle-archive
processor to true
.
If you want to archive also Jingle requests incoming by <message/>
stanzas you need to set archiveJingleMessage
of jingle-archive
processor to true
.