Enabling the enablers

Flight deck

One of the greatest joys we have is the ability to help someone turn a difficult task into an easy one. As we’re running more partner workshops with engineering teams, we’re getting a chance to see our user community engage with our workbench. This is both nerve wracking (we had ambitions, were they met?) and exciting (will these engineers find feature xyz as exciting as we do?).

Developers are inspired by using their skills to solve thorny problems. In doing so, they create a world that is better for whoever their end user is. That’s no different for the team at Recursyv – our tooling is designed to help solve the thorny problem of integrations.

Providing a powerful toolset

The Seamless workbench is something we’ve blogged about before. Since then, it has evolved considerably. The workbench includes a series of headline features which we see as significant enablers for a team developing integrations.

A number of “data pipes” can be setup, each with different sync options.

Imagine a data pipe as the end-to-end connection from System A to System B. Under the hood, Seamless separates source and target with the use of a message queue. This provides additional technical flexibility for sophisticated integrations. In business terms however, it can be useful to imagine these pipes running from System A to System B.

Typically there are a handful of these data pipes, more accurately called Service Buses, which are configured within an integration. Data pipes (service buses) are separated for a number of reasons. One of the most common is that there may be different sync frequencies, e.g. reference data is synced once / day whereas transactional data is synced every 5 minutes.

Any behaviour of the integration can be configured.

The Seamless service is architected into a number of components. Each of these components can be amended by adjusting the config options within the relevant area of the workbench. Once the basics of an integration are setup, the most common configuration items to be set are sync frequencies and data transforms. Sync frequencies are set by twiddling the relevant knobs and dials in the workbench. The query, data transforms and data loads can be specified within the workbench using industry standard XSLT.

Additionally, if the core features of the service do not enable a developer to meet an atypical requirement, the integration can be extended using .NET. The binary can then be included into the integration by specifying it in the config. Extending the feature set of Seamless using custom plugins does not stop developers from using any of the existing Seamless features. These are enabled by default and can be disabled in the config as required.

The location of data and config is setup as a set of variables.

Sometimes it is the simple things that make life easier. Within an integration, it is necessary to store different types of data in different contexts – the config itself needs to be stored, data in transit, service logging, etc. There may even be a need to separate message headers (e.g. a ticket identifier, being stored in a message queue) from more sizeable data items (e.g. images as attachments on the ticket, being stored in a blob store).

The ability to setup a number of storage locations and use these locations as settings throughout the config means that changing these locations is easy. Change the underlying location setting and everything referring to that location knows to work elsewhere. The most obvious time that this is useful is when moving integration configurations from development through testing environments and into production. For each promotion of the integration, there is only a need to update a single location setting.

Integrations can be tested within the workbench.

The workbench allows engineers to operate the entire integration. It’s as simple as clicking a “play” button. Each sub-component (publisher or listener) can be run independently. There are options which can be set to run a few iterations and to introduce delays between the runs.

This allows engineers to test integrations. Within the workbench, the data in transit can be set to be unencrypted allowing developers to test thoroughly. Engineers can monitor data transformations and lookup references when preparing data for the target systems. Logging options, both level of detail and output location, can be set and adjusted as work with the integration progresses.

Recursyv provide a growing number of integration templates.

The Seamless ecosystem includes connector plugins for a number of well known applications. Additionally, Recursyv provide partners with a library of data mapping and transform templates addressing common system scenarios, e.g. Autotask <-> Dynamics 365, Freshservice <-> Dynamics 365, etc.

A template is simply an integration config xml file with mapping and transforms already specified coupled with documentation detailing the settings. The xml file can be loaded into the workbench and, with the relevant connection details specified, set to sync data within a few minutes of launching the workbench.

An integration can be packaged into a standalone config file.

One of our consistent headaches when working with other tools was that there was always a need to pull together a number of different files when sharing an integration configuration. The workbench was designed to address this frustration – the entire integration, including relevant binaries (e.g. connector plugins) – can be packaged into a single XML file.

Quick to sync with a myriad of underlying options

Whenever we’re sharing our integration tooling with engineering teams there are two underlying takeaways that we try and demonstrate: it is quick and it is heavily configurable.

Getting data to sync is really quick. The workbench provides all the tooling to get data moving back and forth within a few minutes. In the growing number of scenarios where templates are available, opening the template into the workbench may get you 90% of the way to meeting the integration requirements within a few minutes.

The integration is heavily configurable. There are config options for a myriad of requirements enabling engineers to meet a wide range of business and technical requirements. Options exist to specify storage locations, sync frequencies, encryption and compression options, change detection and so much more. Data manipulations and transforms can be specified (and re-used). Additionally, where there is something specific that cannot be met using the core service, it can be extended without the need to compromise any of the service features.

Recommended Posts