Well, to put it bluntly: journal is a kind of log, but a very special one. It differs from standard technical application logs in many ways:
- Journal should record logical (sometimes called business) activity inside the system. It can be said to be at a higher level of abstraction than standard technical logs. For example, information about user successful login belongs to journal, but information about result code from authentication server belongs to technical log.
- Content of journal records is defined by analyst, not by programmer. It doesn’t have to contain technical details needed for final problem resolution but it should record all important changes in system.
- Records can be referenced between each other, not like a standard log where there’s only time sequence. Referencing records create user activity or system events tree what can be interesting for later presentation. It brings a concept of a “context” to single record.
- If journal is intended for non-technical audience, its presentation must be comprehensible. Often there’s provided a custom-made viewer with respect to concrete content specifics. Content can be presented in more than one language.
- Journal usually has a longer retention than a standard logs.
- Journal is often used for audit purposes or as evidence in compliant procedures.
So why it is important to journal application events? Mainly because it is an important source of information for customer support about what users did. The better it will be, the faster and more competent will be customer support. Customers and customers of your customer will be satisfied. And as a side-effect low-level support will have less investigation work.
From the above is obvious that it is necessary to think about journal during analysis. Creating journal records should be a natural part of designed processes.
Think like a…
| Analyst think like a Customer support | While designing processes in a new or existing system, think like a staff from Customer support department. Or better, if you have that possibility, ask them what kind of problems did they solve and what information were useful or they lack. |
| Analyst think like a Customer support again… | If you use custom journal viewer, design it to be user friendly even it is for internal stuff. It has a huge effect on your colleagues which will use it day by day. |
| SW architects think like a Customer support | It is tempting to use something existing as a journal viewer (e.g., ELK stack). But does it meet the requirements and daily routine of Customer support department? Don’t leave them out from decision making. They are here the final users. |
| SW architects think like a System administrator | Don’t forget on journal viewer while creating architecture. It has it’s specifics like a bunch of data which must be available and searchable during longer period than technical logs. Also, audience of journal is different and access must be granted. |
Give feedback
| Programmer | Journal records often tends to be too fat. It can affect performance to load all required data to journal them. Talk to Analyst if you find such situation. This is hard to pretend it during analytical phase. |
| Customer support | Report your experiences with using journal and journal viewer to Analysts and System administrators. They need your point of view to make system better. |
| Manager | Do you encounter a long time for compliant resolution? Insufficient information for Customer support could be one of reasons. Ask Customer support about reasons. |
Sensitive content
Underestimation of journal during analysis or additional implementation into existing system tends to use generic approaches to create journal records. For example, the use of aspects bound to classes serving entity storage is very appealing. First implementation is usually very fast, but later problems erase this advantage. Such quick implementations usually lack business logic details and references to other entities what degrades the meaning of journal itself.
Content of journal should be selected carefully. It must not contain information which can lead to misuse of system or act on behalf of another user like passwords, full access tokens etc. If event with such information is made, it is a good manner to record it to journal but sensitive data must be obfuscated or better not included at all. E.g., it’s good to journal when user change its password but password itself must be omitted. In this case user identification and time are more important.
Depends on a kind of software you create; journal usually have a wider audience than regular technical logs. This should be taken into account when selecting journal viewer and designing processes at customer support. Event viewing journal should be journaled. Access to journal content should be authorized and various levels of details can be provided.
Anonymization
A lot of countries provide legislative or regulations trying to enhance individuals’ control and rights over their personal information and to simplify the regulations for international business. If you live in Europe, you probably know GDPR, or CCPA in California USA. This can be complicated by other laws which e.g., in bank industry require data archiving for a long time. Unfortunately, it seems that there’s no generic solution and anonymization process must be custom designed for your system and type of data. It is good to think of it from the beginning because additional implementation can be cumbersome.