Expressions in integrations: RegEx

Regular Expression or the so-called RegEx is quite popular and almost every IT specialist is using it in some form. However, not everybody knows that RegEx is also often used in integrations. Lets see what use cases we apply it for and how we made it super easy even for non-technical users to work with the specific RegEx language in ZigiOps.

What is RegEx?

Lets start with explaining a bit about RegEx itself. A regular expression, often called a pattern, specifies a set of strings required for a particular purpose. In most cases these patterns are used by string-searching algorithms to perform find or find and replace operations on strings, and also for input validation of data.

RegExes are widely supported in programming languages, text processing programs, advanced text editors, databases, and more. There is a specific language or syntax that you need to use in order to form regular expressions.

RegEx in integrations

Regular expressions can be very useful for setting different requirements in integrations. For example, RegEx is a standard way to match and extract text or data that we need to transfer from one system to another. One more example is if we have some kind of a log message, we can search through RegEx the word error and accordingly we will extract only messages that are related to errors.

However, the RegEx language can sometimes be quite complex and not that easy to users with no technical background. This is why in ZigiOps we have pre-defined the most popular RegEx patterns and they are readily available for the end users without the need to know the code or specific language of the RegEx.

We have ready-to-use the following patterns:

• To Upper case
• To Lower case
• First n characters
• Last n characters
• Replace text

All these are available for use in ZigiOps, but if someone wants to define their own custom RegEx this is possible, too, through the Pattern expression.

This way we make it easy and user friendly, but we also provide the freedom to the advanced users to write their own expressions if they want to.

Using RegEx in ZigiOps

Here is how we can define a RegEx or a Pattern expression in ZigiOps.

First, we need to give it a name  for example testregex. Then we need to choose a source field which we will take the data from  for example Summary. After that, we can write different RegEx patterns with the specific syntaxis for that purpose.

In ZigiOps we can use all types of RegEx syntaxis. Our pattern can be anything we need for our purpose or use case. For example .* would search everything.

RegEx is also used to match a certain type of structure in the text, for example an IP address. The IP address always consists of 4 numbers with dots between them. There are also other validations (they cannot be bigger than 255, etc.), but for the example we can use a pattern that matches numbers, dot, numbers, dot, numbers, dot, numbers. This is a RegEx pattern that we often use and it looks like this:

^\d+\.\d+\.\d+\.\d+$|^[-\w]+

It checks if a given field matches the syntaxis for an IP address. The vertical bar means or, and then we get the first word or some type of special character. The idea of this specific RegEx is to extract either IP or the first word from a given field. This is used mostly when we want to extract topology, and more specifically hosts. The different systems usually provide a host name, but this host name could be in different formats. It could be an IP or a short host, or an FQDN (Fully Qualified Domain Name).

If we want to do that in ZigiOps, we can name the pattern shorthost, get it from the field hostname, and with this specific pattern:

• it will first check if the value of the filed is an IP
• if it is an IP  it will get the IP
• if it is not an IP but a short host  it will get the short host name
• if it is an FQDN it will again get just the first part (the short host name)

This way we will always have the short name. This is done in cases when the target system where we report the data wants explicitly just the short name in a specific field.

Then in another field we may need the full hostname (the FQDN). To get that we just need to form another expression that gets everything from the field with .*.

Example use cases

Dynatrace host-up topology to OBM topology

In ZigiOps we have out-of-the box templates for popular integrations, so we can easily load and save our integration in a few clicks. One of our ready to use customizable templates is for transferring host-up topology from Dynatrace to OBM topology.

We can see that in this template we have 3 patterns.

‍

The first one is for the short host name that we just described.

The full host name we get from the "discoveredname" as we know that it always returns either IP or the FQDN format. So with this second pattern we get everything.

In the Mapping of the integration, we want to report the host name. We provide the short hostname because according to the OBM best practices in the name (of the host) field we need to provide either short host name or IP address, but never the whole host name. Then in the primary_dns_name field we need to provide the full name.

‍

Dynatrace metrics to OBM metrics

Dynatrace provides the metrics in a format starting with a specific string builtinapps.web (name of the metric) :names This is quite complex, and we prefer not to use it. For example, here is the full format for a metric called apdex.userType:

We want to simplify the whole string and extract just the part that we need. This is why we use the following pattern:

^builtin:apps\.web\.(.*):names

This pattern says:

• start searching from the beginning
• then look for that string (we escape the dots with a slash  this is a part of the RegEx syntaxis)
• then in the brackets we give the group we want to extract
• and then the ending of the whole string

In the website RegEx101.com, we can check all RegEx patterns. So we can test how our pattern works. We copy the pattern and see what it will return. In Group 1, it returns apdex.userType, and this is what we are actually looking for. And above it we see the whole match.

In ZigiOps RegEx always returns what is in Group 1, so if there is a Group 1, this is what we will receive. If there is no Group 1, we will receive the full match from the RegEx pattern.

If we remove the brackets, that are used for grouping, we will get the exact match (the whole string).

For example, if we change the metric with some other metric, it will return the whole string again. If we return the grouping (the brackets), it will only show that part which is the name of the specific metric. This is how we search metric names in this string.

For almost all Dynatrace metrics we use this type of RegEx.

AppDynamics node metrics to OBM metrics

Here we see again grouped metrics, but in slightly different type of pattern.

We are extracting the metric name with this pattern:

.*\|(.+)$

Again it is specific for the system  it could be before or after a dot.

Then we can use the extracted data in the mapping of the integration. Here, we can see that all values that are in red are actually taken from RegEx expressions that we have pre-defined.

This makes the setup more convenient and cleaner.

The RegEx is validated during the transform phase of the integrations and can be used in the mapping or in the Respond mapping, but it cannot be used in triggers. It is not necessary, as we can always define a condition, based on RegEx, and report the specific data we need this way. Only the Last Time expression can be used in triggers.

In conclusion

RegEx is one of the most used options in integrations and in ZigiOps integration platform. It works for very simple tasks, as well as for more complex data dependencies.

With RegExes we form different patterns, and with these patterns we extract specific data that we need. After that we use the extracted data in the mapping of the integrations. This strategy is quite powerful and can be used for a variety of different purposes.

In ZigiOps we have pre-defined the most popular RegEx patterns and they are readily available for the end users without the need to know the code or specific language of the RegEx.

There are more advanced applications of RegEx, and if you are proficient in it, you can use it to find very specific information within texts or strings and form complex conditions. The ZigiOps users are free to do that through the Pattern expression and this way have really advanced and deep integrations ready in minutes. Many of our clients tell us that they are using patterns and they are happy that they can apply the most popular ones with one click. This way they can match super specific conditions easily and scale their integrations quickly.

If you would like to see how patterns and all the other ZigiOps capabilities can help you in your specific integration use case, book a free demo and we will be happy to show you how it works.