OpcDbGateway

OpcDbGateway is a sofware application platform for application integration, configuration, programming, debugging and cyclic ((IEC 61131-3) as well as event driven (IEC 61499) data processing.

OpcDbGateway can be used to run external programs, scripts and built-in database procedures, generate reports, alarms, log files, OPC tunneling, OPC redundancy providing, SMS and E-mail sending, as part of the extensive SOA systems , as a universal OPC server/client, as a universal application platform with a number of predefined functionalities and options complementarity, and also as an EDGE solution for the Internet of Things (IoT).

It consists of configuration and runtime applications.

The configurator serves to create an application configuration and to save it to a configuration database from which it is then loaded and executed by the runtime application. The configurator allows you to greatly reduce the workload using configuring by dialogboxes instead of programming. It supports application creation efficiency with various support and tuning tools. It includes wizards to quickly configure connections to external applications and process databases to generate historical trends. It is a basic client application for the runtime application with the ability to monitor data points, operate alarms, view log files, defining SQL commands over process databases and viewing their results.

The runtime application consists of three main parts (Figure 1):

        
• Client Modules/Interfaces for communication with external server applications - eg OPC DA client for communication with external OPC servers, DDE client that allows access e.g. to MS Excel files, database client interfaces to access databases such as MS Access, MS SQL Server, MySQL,
        
• Server Modules/Interfaces to communicate with external client applications, OPC server providing data for external applications (e.g. visualization) via OPC DA and AE communication interfaces, as well as web services using OPC UA and OPC XML DA standards,         
        
• Data Processing Module for executing configured or programmed algorithms that is expandable by custom program modules - Dynamically Linked Libraries (DLLs).              

In addition to the client/server architecture, the runtime application can also be used within publish/subscribe architecture (Please see videos).

Figure 2: OpcDbGateway - Functionality, Runtime and Integration of Applications. /click on individual figures/

Although communication via the OPC client and server interfaces as well as through the database client interface is most commonly used (hence the title OpcDbGateway), various other communication interfaces implemented as either an extension DLL, running in the process of the runtime application or as separate processes (applications) with which the OpcDbGateway runtime application communicates via standard inputs and outputs. This allows you to use a wide range of open source applications implemented as standard console applications using languages as C, C++, Python, JavaScript in Node.js or implemented in different languages and deployed using the Docker container.

The runtime application core is an OPC server that provides access to process values. It can be started (and therefore the entire runtime application) from an OPC client application. The runtime application can also be run as a Windows service. Since the configuration application has own OPC client that can start the OPC server in the runtime application, it can read and write data processed by the runtime application. Using a runtime and configurator applications pair the OpcDbGateway is therefore capable of implementing an integrated application with extensive basic functionality, including a basic user interface. This method of operation is especially used to develop and debug an integrated application. More often however, the runtime application is used with customer client applications containing the OPC client and application-specific user interfaces. Client application can also be web based running on a local web server and communicating with a runtime application via the OPC DA interface. However, it is also possible to use a client web application (e.g., cloud-based) communicating over the Internet through the OPC XML DA or OPC UA interface. Together with OpcDbGateway is shipped example of such a web-based client application.

Cyclical and event driven processing

The runtime application offers functionality similar to the PLC. The PLCs typically perform cyclical processing consisting of (1) reading input values ​​from the input modules, (2) their processing and (3) updating the values ​​on the output modules. Both input and output variables or data points are normally read or written in OpcDbGateway from/to one or more devices by the OPC interface as OPC variables. Their values ​​are mapped to so-called Process Values​, which provide an internal representation of an image of a controlled/monitored process. E.g. data from a temperature sensor represent one input data point, setting of a valve in an external device represents one output data point. Each of them is mapped to one process value in the memory of the OpcDbGateway runtime application. In the application configuration, the process values ​​are referenced by memory operands which are used as input and output arguments of configurable internal commands. In cyclic data processing mode, at the beginning of the cycle, the data is first read through the OPC interface or through another communication interface and stored in process values. Subsequently, data is processed according to the configured algorithms. At the end of the cycle, output data is transferred from the process values ​​via the OPC or through other communication interfaces to external applications or devices. The cyclic processing repeatedly performs the same configured algorithms over process values.

Besides cyclical processing, the runtime application can also provide an event driven data processing in that when activating a trigger a data processing algorithm that we will call event is executed. There are three types of triggers: of the type value (related to a process value), of the type time (single or cyclic) and the combined one - time and value. Trigger activation evaluation can be synchronized with cyclic data processing. Such events are referred to as synchronic. Each event has the assigned priority to place it in the priority queue when the event is activated by assigned trigger. Synchronous events are sent to the synchronic queue always at the beginning of the cycle. Other events are routed to the asynchronous queue continuously - as soon as their triger is activated. If more than one event is stored in the queue then they are performed in order of their priority.

Function blocks

Processing algorithms are implemented within the Function Blocks (FB). Cyclical data processing is performed in a single FB named main. In this FB, can be other FBs nested (called from him) up to the depth of 15. FBs contain configurable commands that perform various operations over memory operands, database operands or constants. These operations can be very simple, for example, addition of two process values, more complex, e.g. execute configured SQL command over a process database or calling a user program module (e.g., a PID controller) implemented in the DLL. Asynchronous events also provide either FB execution, call to external process/application or some special operations such as create a new log file.

In addition to FB performed cyclically or in events, OpcDbGateway has standard FBs executed at once at the start, restart, or termination of the application. In the FB, called Start, it is a good idea to initialize the process values from configured constants. This is especially important for those operands that are mapped to inputs and outputs of external resources. Before their first communication with these resources is their state undefined. In FB, called Restart, it's usually not necessary because the initialization is done from data that was stored before the application was shut down. However, there may be situations when it is appropriate to use them for reinitialization.

The difference between process values and memory operands.

1. Process value is a structure that contains the identifier, data type, value, time stamp, timestamp type (internal or external), and information about the quality of the stored data
        
2. process values are stored in the memory of the structure and can be accessed through the field index.
        
3. The memory operand can reference one process value through its index
        
4. memory operands provide process values for their use in the role of input and output arguments of configurable commands
        
5. memory operands can be placed in a tree structure configuration which provides the ability to structure the application appropriately
    

Database operands and constants

Similar to memory operands that access data points from external applications, OPC servers, database operands make available entities such as tables, columns in tables, individual items in tables for use as arguments in configurable commands.

Configurable constants can also be used as arguments in configurable commands. It is advisable to use them, for example, for initializing the process values ​​so that they are not in the undefined state at a time before the first input/output operation. For initialization purposes, however, we can also use constants pre-stored in process databases.

In addition to database operands, the runtime application can work with process databases through parameterizable database queries and parameterizable user reports. Parameters are memory operands, and in the result of the operation the actual values ​​of the related precession values ​​are used. Parameterizable messages can be written either into log files or into process databases.

Configurable commands

Configurable commands (CC) can be compared to C-language void functions that have maximum two inputs and one output. Inputs and outputs are references to named entities - constants, memory and database operands, named user reports and database queries. E.g. memory operands refer to process values through their index. The naming allows you to select function parameters using dialog boxes. Configurable commands allow you to perform arithmetic, logical, comparing, database, database statistics, predictive operations as well as storing user messages in a database or log file, call of nested function blocks and program modules implemented in the DLLs. Using numbered configured commands are created FB. The order of execution is ascending according to the CC number. When configuring FB, it is advisable to leave gaps in the numbering for possible insertion of CC. When performing some CCs automatic data type conversions are performed if needed.

Multithreaded functionality

The runtime application uses multithreaded functionality. The Windows operating system on which it is running has the ability to automatically divide the load between the individual kernels of the multi-core processor. The 1st thread of the runtime application provides an evaluation of the state of triggers, the 2nd thread provides processing of synchronous operations, the 3rd thread is used for asynchronous operations. A special thread is started to execute especially time-consuming configurable commands. Each user DLL can be performed either in the custom thread or in the synchronous thread. In addition, both the OPC server and the client use several of their own threads.

Cyclic (IEC 61131-3) and Event (IEC 61499) functionality

The runtime application functionality makes it possible to use the principles described in IEC 61131-3 and the newer IEC 61499. The currently prevailing IEC 61131-3 standard does not distinguish between events and data, and its approach is therefore more appropriate for cyclical data processing as in the FB main in OpcDbGateway. Within the IEC 61499 architecture, FB is a universal design artifact which, unlike FB according to IEC 61131-3, has, in addition to data inputs and outputs, the event inputs and outputs. FB can be used not only to describe the decentralized control logic, but also to describe the properties of devices such as, for example, their interfaces. Thus, the complete functionality of the distributed control system can be represented by FB and interconnections between them. The cyclical execution of the PLC runs all the data blocks associated with the data on arrival. In contrast, according to IEC 61499, FB execution is event-driven and therefore every function block does not have to be performed in each cycle.

Although the OpcDbGateway was not strictly developed based on IEC 61131-3 or IEC 61499, cyclical execution and event-driven implementation make it possible to use the principles defined in these standards for distributed applications.

The distributed control architecture according to IEC 61499 defines the key elements of the distributed control system. These are apps, devices, and resources. An application is a related set of functions that must work together to accomplish the control task. The device is a control unit with one or more processors. The device is connected to physical inputs/outputs and also communicates with other devices in the network. We usually consider devices that consist of a single PLC, which controls all control functions of the machine. Such a PLC typically has a single processor. PLC with its I/O and network interface is a device. The PLC processor is the resource. The CPU program is an application. Such a system is therefore a device with only one resource and one application. However, multiple applications may be available for such resources.

OpcDbaGateway is a resource on which multiple applications can run. I/O data from various external devices can be mediated e.g. using external OPC servers or other software communication drivers that can run either on the same computer or on another local network or even connected over the internet.

The meaning of the functional block concept in OpcDbGateway and IEC 61499 differs. FB to IEC 61499 is an abstraction that can be used to model distributed applications using FB. (You can then implement it with use OpcDbGateway.) This interconnection is implemented at both the data level and the event level. The data outputs of one FB are linked to the data inputs of another FB. Performing one FB is initiated by event generated from another FB.

The OpcDbGateway function block in the narrower sense is the sequence of configurable commands. FB in the sense of IEC 61499, however, can be executed with the OpcDbGateway also by calling an external application. The difference is also in the term event. In IEC 61499 event represents any variable activation value that initiates the execution of some type of activity in the FB. In the OpcDbGateway, a configurable event represents some sort of configurable activity as FB, external application, special activity directly implemented in a runtime application such as creating a report, checking disk space to store reports and log files. This event is started by a configurable trigger. A configurable triger is either a value of an activation memory operand or a defined time. The activation memory operand of the configurable triggger is thus possible to identify with the event in accordance with IEC 61499. FB breakdown according to IEC 61499 can be configured in OpcDbGateway as follows: (1) we select an activation memory operand that we assign to the configured trigger. (2) Configure the event of the type "call FB" of. (3) In addition to the main functionality, in the given FB, the process variable referenced by the memory operand that is configured as a trigger of the type value is set to the status "activated" and so next chained event of the type "call FB" is started.

In OpcDbGateway, one triger can start quasi paralleling even multiple events. Their order of execution is then given by their configured priority.

However, FBs chaining can be configured also in a simpler way. The CALL configurable command allows you to call FB from another FB when the defined memory operand gets an activation value. This memory operand can also be identified with event in the sense of IEC 61499.

Environments tu run the OpcDbGateway runtime and providing interoperability

It is possible to launch OpcDbGateway on any device that can run MS Windows. That means not only standard but also industrial computers and embedded systems. Interoperability of this system with many other platforms and operating systems is ensured by utilizing web services according to OPC XML DA and OPC UA specifications or by custom drivers implemented in enhancing DLLs or some of the communication drivers (such as MQTT) that provide the publish/subscribe functionality. OpcDbGateway can use multiple database systems (Figure 3) also within one integrated application.

Alarm Systems

OpcDbGateway implements two alarm systems – basic, based on the implementation of OPC AE 1.1. standard and enhanced one that allows many operations to be performed based on various events, for example logical operations above multiple input variables, alarm status logging, database operations, etc. It is possible to deliver a web application to visualize the functionality of these alarmsand alarm log files archive. Creation of new log files during a defined time or within a defined time period can be configured.

Data persistence

OpcDbGateway ensures data persistency – that means, after a restart, if required, it uses data stored before application shutdown.

Guarding own resources

OpcDbGateway monitors the usage of it's own resources such as virtual memory, disk space for logs, reports, database and power source.

Integrated Application Configuration Tools and Procedures

Wizard to automatically map external resources to the internal process image

The first step in creating an application in OpcDbGateway is to create a structured data model of the application by mapping both external and internal data points to the tree directory structure of memory operands. Directory structure of memory operands is possible to create by manual configuring in the configurator, but because it is often necessary to map very large number of data points in practical applications, it is very useful to use built-in auxiliary tools.

Many external data points are accessed through external OPC servers already configured within tree structure of their own address space. The OpcDbGateway configurator can connect to such OPC servers via its own OPC client, and retrieve selected parts of the address space tree structure to the the tree directory structure to place the memory operands - Figure 4.

Use the wizard to create so-called OPC groups with different parameters for data transmission, with read/write settings, for setting whether periodic data transmission or data transmission based on notifications and setting of parameters of reconnection in case of interrupted connection with an external OPC server has to be performed. Working with this wizard can be tried using the OpCDbGateway supplied simulation OPC DA server SAEAutomation.OPCSimDA.

A similar wizard was created, for example, also for accessing DDE servers.

Wizard for automatically configuring client application access to the internal process image

For client applications, access to process values (internal data points) is secured through the OpcDbGateway internal OPC server. However, the address space of this OPC server must also be configured using the configurator OpcDbGateway. This also has not to be done manually but automatically. This activity is included in the above mentioned wizard for mapping external OPC servers address space. When creating an internal process image, we can automatically create a link to the OPC items of the internal OPC server  for those process values that have a direct link to external data points.

Wizards for access to databases

In order to work with data from database tables using configured or programmed OpcDbGateway features, it is necessary to provide mapping of used database tables from the database to the configuration or in the opposite direction from the configuration to the database. It is very important to have the same table definitions on both sides. For this purpose, you can use wizards for creating the connection string to an existing database, a wizard for mapping configured tables to an existing database and a wizard for maping of existing database tables to database operands. These tools provide the ability to integrate various database applications with the OpcDbGateay runtime application.

Figure 5: Software PLC (Soft PLC) uses memory and database operands. /click on this figure/

The configurator also lets you view the contents of individual tables, or define different views on databases using SQL commands created directly in the configurator.

Creating an Application Template

When you create a functional application base using the Configurator main menu and the File-> New command, the internal data model part of the application's performance data monitor is created automatically. System Variable Field of Process Variables (indexes 1-100) is automatically mapped to the address space of the internal OPC server. From the OPC client in the configurator, we will connect to the internal OPC server of the runtime application using the View-> Monitor->View command. A new look is displayed in the configurator, in which nothing is displayed. From the tree structure of the application in the configurator we choose Internal OPC Server-> Address Space-> System->Status. Several OPC items, such as those listed above, are displayed in the monitor window. System.Status.LocalTime. This item will be updated periodically. So we've created the first functional app, but nothing useful yet performs only monitors the performance of the runtime app.

Figure 6: Displaying of system variables in the OPC client built into the configuration application. /click on this figure/

When creating an application template, an MS Access database process is automatically created as well. We can immediately begin using it, adding custom tables, or adding/replacing it with other process databases of a different type. Process database contains several predefined tables, e.g. to save the current status and alarm history.

Next steps when configuring an application

In addition to automatically mapped external data points, we will also need to manually create memory operands for inbound results from configured commands and for communication with user DLLs. When they are created, they are automatically offered freely usable process value indexes. The individual memory operands can be accessed from user's client applications via an internal OPC server of the runtime application. This is why the "Generate OPC Item" button is in the dialog box for defining of memory operands. It is possible to automatically create an item in the address space of the internal OPC DA server. The memory operand is usually located in a directory structure of memory operands, and this directory structure is automatically mapped to address space of the internal OPC server.

The next step in configuring an application is to create functional blocks to implement each application's functionality. The application may contain either only the cyclically executed functionality, event functionality only, or a suitable combination thereof. In the case of purely cyclical functionality, all functional blocks are nested into the cyclically executed function block main, with the exception of one-time functionality for initializing and terminating the application. It is possible to configure the period size for cyclical execution so that all FBs can be executed within one cycle. System variables that allow you to monitor both synchro and asynchronous event queue sizes should never grow smoothly when you run the application. If it would  such a situation is necessary to increase the period of cyclical execution or to modify the function blocks.

Graphical Configuration View

The configuration structure consisting of individual function blocks is possible to let show graphically (Figure 7) in the configurator. Double-clicking individual blocks in the graphical view you can go directly to their editing using the corresponding dialog boxes. From the graphical view it is possible to see which activities are performed periodically (this is only FB main in which other FBs are usually nested) which activities are performed during the start and end of the application, as well as activities that are performed on the basis of configurable triggers and events.

Testing OpcDbGateway configuration

After a project is configured with the help of OpcDbGateway configurator,a syntax check of the entire configuration can be performed (Figure 8). It is possible to jump to the related dialog box to remove a configuration error directly from the list of detected errors.

Figure 8: Configuration checking. /click on this figure/

Figure 9: Log file viewer. /click on this figure/

Running application can be tested in two ways: (1) using Configuration Built-in OPC Client (Figure 8), which is capable of monitoring the states of all OPC internal OPC variables, including system varialbes that tell you about app activity or (2) using messages written in external logs ( Figure 8 ). The range of information logged into log files as well as their size is configurable. It is possible to record either details of each operation performed, its duration, or just the error states of the runtime OpcDbGatway application. Into log file also user-defined messages can be written.