Category Archives: Configuration

Tips,Tricks and Hints to configuration

Sling Context-Aware configuration (part 5): thoughts on a production layout

In the past blog posts of this series I outlined some of the capabilities of the Sling Context-Aware configuration. I was focussing on the demo aspect of it, and ignored aspects of it which are relevant when we talk about the use of it in an production environment.

(Note: There is not too much experience using Sling CA Config in wider production use. In this blog post I discuss these based on my experience with the product I gained in many projects; and I assume that it will work. If you have different experience with, I am interested to hear from them!)

For example I placed the configuration editor in /content/ca-config/en/configuration-page, while the configuration itself was located somewhere in /conf. Also the WCM.io editor can only edit the configuration for the context it is located in; that means, that your editor at /content/ca-config/configuration-page cannot modify the configuration applicable to the content context /content/ca-config/en, but you have to have a dedicated configuration editor located in that context.

Let’s stick with /conf in this blog post . Every code and every user which needs to consume configuration has to have read access to /conf; and to make inheritance working, you have to have read access to the configuration you are inheriting from as well. Generally spoken, you need to provide read access to a large part of /conf to all users. Which is typically not a problem.

But if you want to prevent access to certain parts of /conf but still be able to use them as part of Context Aware Configuration you have to trick a bit.
Imagine the case, that you want to store password data for certain backend system in CA-Config. You want to make them read- and writable only to the group which administers this backend access, but not for other users. But nevertheless all users should be able to use this backend service.

In that case the best way is to hide the access to the backend system in an OSGI service which utilizes a service-user to access these CA-Config resource in /conf. Whenever an user access this backend system, he will utilize this service and its service-user to read the configuration, and therefor this region of /conf can be locked down.

In the next blog post I want to show you another aspect of “making CA-Config production ready”: Publishing configuration.

Sling Context-Aware Configuration (part 4) — using inheritance

Till now this series of blog posts has only worked with configuration which has been explicitly set. But when working in hierarchical systems (like the AEM content tree) it’s often useful when you can inherit things from a parent container/concept. That means, that a specific setting on a node has effect on a complete subtree below that node instead of just the single node.

And of course Inheritance works with CA-Config as well. For easier demonstration I updated the demo content at https://github.com/joerghoh/cqdump/tree/master/ca-config-example in a way, that we can edit the configuration at 3 different locations:

  • /contnet/ca-config/configuration.html
  • /content/ca-config/en/configuration.html
  • /content/ca-config/fr/configuration.html

We always use the WCM.io configuration editor I introduced in the last post.

For demo purposes I also created a new configuration “de.joerghoh.cqdump.caconfig.inheritance.InheritanceDemoConfig” (code). Be aware that the configuration editor only works when the Pojo classes for the configuration are properly annotated.

Ok, let’s get started.

Open the configuration at /contnet/ca-config/configuration.html, hit the “add” button at the top and select the “Configuration to demo the inheritance” configuration. Then change the value of the “aProperty” configuration, check the “enable property inheritance” box and press “Save”.

caconfig-inheritance-globallevel

You have created a configuration which should be inherited down to the other levels. Specifically we should be able to see it at /content/ca-config/en/configuration.html. So let’s check it.

caconfig-inheritance-locallevel

On a repository level only the property “sling:configPropertyInherit” has been added to /conf/ca-config/content/sling:configs/de.joerghoh.cqdump.caconfig.inheritance.InheritanceDemoConfig.

caconfig-inheritance-crxdeview

It is that simple to allow configuration inheritance. And there are many usecases where this can be useful.

If you look back, we can build this functionality also the InheritanceValueMap. But right now we only scratched the surface of the CA-Config functionality. But before we dig deeper into it, I will discuss some aspects of Sling CA-Config for production usecases.

Sling Context-Aware configuration (part 2)

Update (April 30th, 2018): I changed the name of the configuration item to align with with future development. Next to this change I also added a few bugfixes in the git repo.

In the last 2 blog (1,2) posts I talked about why a more content-oriented and content-centric way to store configuration is useful. And I also described 2 scenarios in which CA-Config can actually be used. Now I want to demonstrate how Sling Context-Aware Configuration can actually be used.

Let’s take the example from last blog post: For the approval workflow we want to configure the approval group externally via Sling CA-Config, so it can be configured per subtree/page. I use AEM 6.3 here, but it should work with 6.2 as well.

In the next steps I will guide you through the steps required to implement these steps. For reference the complete project is available at https://github.com/joerghoh/cqdump/tree/master/ca-config-example.

  • Create a structure in /conf where you would like to store your configuration. In my example the content lives in /content/ca-config with the subites “en” and “fr”, and for /conf I choose /conf/ca-config/content/.
  • Create a property “sling:configRef” with the value “/conf/ca-config/content” at /content/ca-config/jcr:content/. This links your content to the configuration.

In the CA-Config lingua you created a configuration bucket at /conf/ca-config/content, which is referenced by /content/ca-config.

caconfig-configRefs

And that’s it. Now every lookup through Sling CA-Config will lookup there.

Now let’s create some code which is using such a configuration. I created a DynamicParticipant WorkflowStep named “CaConfigParticipantStepChooser“, which assigns the workflow to a group defined by CA-Config. As you can see there a POJO class “CaConfigParticipantStepChooserConfig” is used to store the config read from CA-Config.

caconfig-wfstep

When you run this code, it will try to lookup the config property “approverGroup” (which is derived from the field name in CaConfigurationParticipantStepChooserConfig), but it returns the default value “admin”, as we haven’t provided any configuration value yet.

Now let’s assume that this workflow step, if it is executed on content below /content/ca-config, should assign the workflow to the group “content-authors“. Create a folder “/conf/ca-config/content/sling:configs” and add a node with the name “de.joerghoh.cqdump.caconfig.workflow.CaConfigParticipantStepChooserConfig“; add the property “approverGroup” and set it to the value “content-authors“.

caconfig-configsetting

To validate this setting there is a small tool in the OSGI webconsole.

  • Goto localhost:4502/system/console/slingcaconfig (in the navigation it’s “Sling” => “Context-Aware Configuration”)
  • enter “/content/ca-config” as content path
  • Enter “de.joerghoh.cqdump.caconfig.workflow.CaConfigParticipantStepChooserConfig” as “Other config name”.
  • And press “OK”.

 

It will display that that for this configuration name a property “approverGroup” with the value “content-authors” can be resolved. Which is exactly what we want.

When you test the workflow now, the workflow will be assigned correctly to the “content-authors” group.

To demonstrate the flexibility of context-aware configuration, let’s create a new folder /conf/ca-config/content/fr/sling:configs, add a node “de.joerghoh.cqdump.caconfig.workflow.CaConfigParticipantStepChooserConfig” and add a property “approverGroup” with the value “contributors“. Additionally add the property”sling:configRef” with the value “/conf/ca-config/content/fr” to the /content/ca-config/fr page. When you start the workflow now on /content/ca-config/fr the workflow will be assigned to the contributors group. And of course this also works for every page below /content/ca-config/fr.

(An important remark here: The paths are choosen arbitrarily. Especially in /conf you are very flexible, and the structure there does not necessarily need to match the structure below /content. I just choose the same structure for a better understanding of this example, and for convenience reason I recommend you to do the same.)

Congratulations, you are now able to use Sling Context-Aware Configuration! You can now freely package /content to move it to another stage, and the per-stage approver groups do not need to be re-adjusted.

But to be honest, this is a very simple way, and it has some drawbacks:

  • You need to use CRXDE to maintain the properties in /conf.
  • The need to create sling:configRef nodes for all buckets.

Let’s try to address these topics another time, in the next blog post.

 

 

 

Java 7 support for CQ5?

As Java 7 has been launched these days, the question arises real soon: “Does Adobe support Java 7 as runtime for CQ5?”.

So, the clear answer is: No, it isn’t supported. Mostly because of some issues which can cause corruptions in the Lucene index. So, of course, you give it a try and tackle the risk yourself (as you can run CQ5 on a Windows 7 box or on Debian Linux); but don’t complain, if you receive some strange behaviour.

ps: Just adding -XX:-UseLoopPredicate to your JVM parameters won’t solve the problem (according to the Lucene Website).

CQ5 logging

This week I held a workshop at a customer and someone asked me “How do other customers of Day handle their logfiles? Do they check them and analyze the logfiles?” I had to admit that “according to my experience nobody really cares about them. The only situation they care about them is when the disk is full f them.” Yeah, a sad truth.

But this brings us to todays topic: Logfiles and keeping track of them. CQ5 is by default pretty noisy; if you check the file crx-quickstart/logs/error.log after some requests have been made, you see a lot of messages of loglevel “INFO”. Yes, sometimes quite interesting, but in the end they pollute the log and the real important messages vanish in the pure mass of these noise. So, at least for production systems, the loglevel should incrased to WARN or even “ERROR”, so only logs at level WARN or ERROR are logged, INFO is supressed.

So, how can this be achieved? Sling as part of the WCM part of CQ5 brings its own logging, it can be configured using the Felix console and is well documented on the Day documentation site. CRX (at least up to CRX 2.1) does have its own logging mechanisms (log4j), which can be reconfigured in the crx-quickstart/server/runtime/0/_crx/WEB-INF/log4j.xml file.

And, on top of this all, we have on a standard Unix system

  • crx-quickstart/logs/stderr.log and crx-quickstart/logs/stdout.log
  • crx-quickstart/logs/server.log
  • crx-quickstart/server/logs/startup.log

neat, isn’t it? Ok, how can you configure them?

Short answer: you can’t. At least it isn’t documented.

The stdout.log and stderr.log and the standard output and standard error channel of the java process, which is redirected to these files. Especially stdout.log fills up pretty fast, because CRX logs all its messages also to the stdout. So fixing up the log4j.xml file is mandatory, because we don’t need this information twice in the crx/error.log and the stdout.log file. Oh, and of course these files aren’t rotated, but new data is appended only. So it grows and grows and grows.

The server.log file is written by the CQSE servlet engine and cleared when the servlet engine is started. Same as for the startup.log, which contains the output of the serverctl script before starting the java and also error messages, if the java process doesn’t start at all (most times due to invalid parameters).

A few recommendations (just a personal point of view):

  • Log rotation should be performed on a timely basis and not be based on the size of the logfile. You should have enough space then and monitor it closely, of course. But this helps you to lookup a certain problem (“Wait, it was yesterday, so it must be in error.log.0 file”) without hassles.
  • implement your own logfile rotation for the stdout.log and stderr.log files. I fill a bug for it too, but till then you need to help yourself. Sorry.
  • Increase loglevel to WARN. INFO just logs too much noise.
  • Adjust the log4j.xml of CRX and change it to something like this:
<root>
 <level value="warn" />
 <appender-ref ref="error"/>
 </root>

So adjusting the logging according to your needs shows, that you care about them and know, that they are useful at all. Which is a required step to do some analysis on them. But that topic is a candidate for one of the next postings.

Application monitoring vs System monitoring

Recently I was asked how a CQ monitoring should be setup. The question was acompanied by a very short description, how the monitoring was supposed to look like. There were some points like “monitoring the publishing by requesting pages”, “check if the java process is running” and “checking the free disk space”. Obviously they just setup some new servers for this environment and thought that they need to monitor some parameters.

As a first step I advised to separate the topics “application monitoring” and “system monitoring”. One might wonder why I suggest to make a strong division between these topics, so here the background.

Standardization is one of the key topics in IT; everything, what is standardized, can be reused, can be exchanged by a compatible product, and finally lowers the cost. So IT operation teams tend to standardize as much as they can, because as intermediate step to lower costs standardization allows automation.

Basic system monitoring is such a thing. Every computer has componentes, which can be monitored such way: Disk health, CPU temperature, status of the power supply units, internal temperature. But also CPU utlization, free disk space, network connectivity or if the system starts to swap. And many more. These are basic metrics which can be measured and monitored in a consistent and automatic way.

For these points it doesn’t care if the system runs a data warehouse application, a mailserver or CQ. They are all the same and the reaction is really comparable if one of these monitored things fails: If a disk is dead, one needs to replace it (with not-so-old servers you can do this online and without service interruption). The procedure may differ from computer to computer, but the basic action is always the same: When the monitoring shows that a disk failed, lookup the type of the failed disk, get a new one, and go the computer and replace it according to the guidelines of the computer manufacturer. That’s it. You can handle some thousand servers that way with only a few people.

Running applications isn’t standardized that way. One application requires a Windows Server, other run because of their history only on big iron. One vendor offers performance guarantees only for linux systems, and other vendors don’t care about the platform as long as they have a Websphere Application Server as base. Some applications are designed to run centralized, other applications can be clustered. Some have good logging and messages you can use for diagnosis, others don’t have that and error causes must be detected with system tools like truss or strace.
So applications are highly non-standardized and often need special skill and knowledge in order to operate them. Automatisation is a very hard job here, and there must be support by management to get every part of the organisation in the right direction.

(As a side note: In my former life before I joined Day I worked in a large IT operation organisation where every application was somehow non-standard; some less, but also some completly out of every order. IT tried its best to create some kind of standardization, but the busineses often didn’t care that much about it; also developers didn’t knew much about IT operations, so “but it works on my machine!!” and “Just open the firewall, so these 2 components can talk to each other” was often heard in early project stages.)

These applications also need completly different kinds of monitoring. The implementation for SAP monitoring looks different than the application monitoring for a web application. The actions the take in case of problems probably differ even more; and when it comes to investigate on errors the webapplication administrator cannot do anything on the SAP system. And vice-versa.

So it’s advisable to separate the monitoring into 2 parts: The basic system monitoring and the application monitoring.

The system monitoring part can be done by one team for all servers. The application monitoring is too complex and too different, the actions sometime require so often special knowhow, that it must be adjustable to the needs of each application and application administrators.

As a final conclusion: Everytime a computer system is setup, put it into the basic system monitoring. So failing disks can get replaced.
And when the application administrator deploys the application on it, the special monitoring stuff is installed then.
Just because the needs and skills, which it takes to react on monitored issues, are very different.

User administration on multi-client-installations

Developing an application for a multi-client-installation isn’t only a technical or engineering quest, but also reveals some question, which affect administration and organisationial processes.

To ease administration, the user accounts in CQ are often organized in a hiearchy, so that users which are placed higher in the hierarchy, can administrate user which are lower in the hierarchy tree below them. Using this mechanism a administrator can easily delegate the administration of certain users to other users, which can also do adminstrative works for “their” users.

The problem arises when a user has to have rights in 2 applications within the same CQ instance and every application should have its own “application administrator” (a child node to the superuser user). Then this kind of administration is no longer possible, because it is impossible to model a hierarchy where neither application administrator user A has a parent or child relation to application administration user B nor A and B are placed in the hierarch higher than any user C.

I assume that creating accounts for different application but the same person isn’t feasible. That would be the solution which the easiest one from an engineering point of view, but this does contradict the ongoing move not to create for each application and each user a new user/password pair (single sign on).

This problem imposes the burden of user administration (e.g assigning users to groups, resetting passwords) to the superuser, because the superuser is the user, which is always (either by transition or directly) parent to any user. (A non-CQ-based solution would be to handle user related changes like password set/reset and group assignment outside of CQ and synchronize these data then into CQ, e.g. by using a directory system based on LDAP.)

ACLs, access to templates and workflows should be assigned only using groups and roles, because these can be created per application. So if an application currently is based on a user hierarchy and individual user rights it’s hard to add a new application using the same user.

So one must make sure, that all assignments are only based on groups and roles, which are created per application. Assigning individual rights to a single user isn’t the way to go.